2024-03-21T12:33:18.845 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-21T12:33:18.845 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614946 --owner scheduled_gabrioux@teuthology --verbose --name gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi --description orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} -- /home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614946/orig.config.yaml 2024-03-21T12:33:18.877 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T12:33:18.965 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614946 branch: wip-guits-testing-2024-03-21-0818 description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} email: gabrioux@ibm.com first_in_suite: false job_id: '7614946' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-testing-2024-03-21-0818 ceph: conf: client: debug ms: 1 global: mon election default strategy: 1 ms bind msgr2: false ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) mon_bind_msgr2: false sha1: 960fbbe4645c2becc8084669be194c95183c7753 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: cephadm-package install: ceph: flavor: default sha1: 960fbbe4645c2becc8084669be194c95183c7753 extra_packages: - cephadm selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-guits-testing-2024-03-21-0818 sha1: 960fbbe4645c2becc8084669be194c95183c7753 owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 187 sha1: 960fbbe4645c2becc8084669be194c95183c7753 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-testing-2024-03-21-0818 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_960fbbe4645c2becc8084669be194c95183c7753/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 960fbbe4645c2becc8084669be194c95183c7753 targets: smithi043.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMKQr6xJYZYRSZOKdOcsDYj9CfJ0Mdg2xHnOMm5ZlIb3Rp0OPPuqjIHStJ1jsB2FMIOxo3b96kXVOMWkJLEbG/w= smithi155.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLo69eurKAR/4SxHFjnaH8P/uQ75B/82VOaeUJy+kG/DcHJuvyscQ/1DwEBzN7uTxZkbFCRdcSnhyEA9+7bdnfE= tasks: - pexec: all: - sudo dnf install runc -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-21_12:04:17 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-21T12:33:18.965 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_960fbbe4645c2becc8084669be194c95183c7753/qa; will attempt to use it 2024-03-21T12:33:18.966 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_960fbbe4645c2becc8084669be194c95183c7753/qa/tasks 2024-03-21T12:33:18.966 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-21T12:33:18.968 INFO:teuthology.task.internal:Checking packages... 2024-03-21T12:33:18.992 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '960fbbe4645c2becc8084669be194c95183c7753' 2024-03-21T12:33:18.992 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-21T12:33:18.992 INFO:teuthology.packaging:ref: None 2024-03-21T12:33:18.992 INFO:teuthology.packaging:tag: None 2024-03-21T12:33:18.992 INFO:teuthology.packaging:branch: wip-guits-testing-2024-03-21-0818 2024-03-21T12:33:18.992 INFO:teuthology.packaging:sha1: 960fbbe4645c2becc8084669be194c95183c7753 2024-03-21T12:33:18.993 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-testing-2024-03-21-0818 2024-03-21T12:33:19.130 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2267.g960fbbe4 2024-03-21T12:33:19.132 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-21T12:33:19.138 INFO:teuthology.task.internal:no buildpackages task found 2024-03-21T12:33:19.138 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-21T12:33:19.186 INFO:teuthology.task.internal:Saving configuration 2024-03-21T12:33:19.198 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-21T12:33:19.221 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T12:33:19.246 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614946', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 12:27:01.339137', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMKQr6xJYZYRSZOKdOcsDYj9CfJ0Mdg2xHnOMm5ZlIb3Rp0OPPuqjIHStJ1jsB2FMIOxo3b96kXVOMWkJLEbG/w='} 2024-03-21T12:33:19.273 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614946', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 12:27:01.340862', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLo69eurKAR/4SxHFjnaH8P/uQ75B/82VOaeUJy+kG/DcHJuvyscQ/1DwEBzN7uTxZkbFCRdcSnhyEA9+7bdnfE='} 2024-03-21T12:33:19.273 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-21T12:33:19.280 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-03-21T12:33:19.280 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-03-21T12:33:19.280 INFO:teuthology.run_tasks:Running task console_log... 2024-03-21T12:33:19.382 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f75d3cff0d0>, signals=[15]) 2024-03-21T12:33:19.382 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-21T12:33:19.389 INFO:teuthology.task.internal:Opening connections... 2024-03-21T12:33:19.389 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi043.front.sepia.ceph.com 2024-03-21T12:33:19.390 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T12:33:19.495 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi155.front.sepia.ceph.com 2024-03-21T12:33:19.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T12:33:19.582 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-21T12:33:19.589 DEBUG:teuthology.orchestra.run.smithi043:> uname -m 2024-03-21T12:33:19.607 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64 2024-03-21T12:33:19.607 DEBUG:teuthology.orchestra.run.smithi043:> cat /etc/os-release 2024-03-21T12:33:19.665 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="CentOS Stream" 2024-03-21T12:33:19.665 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="9" 2024-03-21T12:33:19.665 INFO:teuthology.orchestra.run.smithi043.stdout:ID="centos" 2024-03-21T12:33:19.665 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE="rhel fedora" 2024-03-21T12:33:19.665 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="9" 2024-03-21T12:33:19.665 INFO:teuthology.orchestra.run.smithi043.stdout:PLATFORM_ID="platform:el9" 2024-03-21T12:33:19.665 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T12:33:19.665 INFO:teuthology.orchestra.run.smithi043.stdout:ANSI_COLOR="0;31" 2024-03-21T12:33:19.665 INFO:teuthology.orchestra.run.smithi043.stdout:LOGO="fedora-logo-icon" 2024-03-21T12:33:19.666 INFO:teuthology.orchestra.run.smithi043.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T12:33:19.666 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://centos.org/" 2024-03-21T12:33:19.666 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T12:33:19.666 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T12:33:19.666 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T12:33:19.666 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2024-03-21T12:33:19.693 DEBUG:teuthology.orchestra.run.smithi155:> uname -m 2024-03-21T12:33:19.710 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2024-03-21T12:33:19.710 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2024-03-21T12:33:19.767 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="CentOS Stream" 2024-03-21T12:33:19.767 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="9" 2024-03-21T12:33:19.767 INFO:teuthology.orchestra.run.smithi155.stdout:ID="centos" 2024-03-21T12:33:19.767 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE="rhel fedora" 2024-03-21T12:33:19.767 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="9" 2024-03-21T12:33:19.767 INFO:teuthology.orchestra.run.smithi155.stdout:PLATFORM_ID="platform:el9" 2024-03-21T12:33:19.767 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T12:33:19.767 INFO:teuthology.orchestra.run.smithi155.stdout:ANSI_COLOR="0;31" 2024-03-21T12:33:19.768 INFO:teuthology.orchestra.run.smithi155.stdout:LOGO="fedora-logo-icon" 2024-03-21T12:33:19.768 INFO:teuthology.orchestra.run.smithi155.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T12:33:19.768 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://centos.org/" 2024-03-21T12:33:19.768 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T12:33:19.768 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T12:33:19.768 INFO:teuthology.orchestra.run.smithi155.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T12:33:19.768 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2024-03-21T12:33:19.792 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-21T12:33:19.801 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-21T12:33:19.817 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-21T12:33:19.817 DEBUG:teuthology.orchestra.run.smithi043:> test '!' -e /home/ubuntu/cephtest 2024-03-21T12:33:19.819 DEBUG:teuthology.orchestra.run.smithi155:> test '!' -e /home/ubuntu/cephtest 2024-03-21T12:33:19.835 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-21T12:33:19.840 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-21T12:33:19.840 DEBUG:teuthology.orchestra.run.smithi043:> test -z $(ls -A /var/lib/ceph) 2024-03-21T12:33:19.876 DEBUG:teuthology.orchestra.run.smithi155:> test -z $(ls -A /var/lib/ceph) 2024-03-21T12:33:20.027 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-21T12:33:20.077 INFO:teuthology.run_tasks:Running task kernel... 2024-03-21T12:33:20.109 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-21T12:33:20.109 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-21T12:33:20.110 DEBUG:teuthology.orchestra.run.smithi043:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-21T12:33:20.110 DEBUG:teuthology.orchestra.run.smithi155:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-21T12:33:20.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T12:33:20.127 DEBUG:teuthology.orchestra.run.smithi155:> uname -r 2024-03-21T12:33:20.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T12:33:20.128 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-03-21T12:33:20.184 INFO:teuthology.orchestra.run.smithi155.stdout:5.14.0-361.el9.x86_64 2024-03-21T12:33:20.185 INFO:teuthology.task.kernel:Running kernel on smithi155: 5.14.0-361.el9.x86_64 2024-03-21T12:33:20.185 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum install -y kernel 2024-03-21T12:33:20.185 INFO:teuthology.orchestra.run.smithi043.stdout:5.14.0-361.el9.x86_64 2024-03-21T12:33:20.186 INFO:teuthology.task.kernel:Running kernel on smithi043: 5.14.0-361.el9.x86_64 2024-03-21T12:33:20.186 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum install -y kernel 2024-03-21T12:33:22.192 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS-9 - AppStream 17 MB/s | 6.7 MB 00:00 2024-03-21T12:33:22.412 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-21T12:33:23.651 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS-9 - BaseOS 6.9 MB/s | 2.0 MB 00:00 2024-03-21T12:33:24.003 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS-9 - BaseOS 4.0 MB/s | 2.0 MB 00:00 2024-03-21T12:33:26.616 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 3.1 MB/s | 8.0 MB 00:02 2024-03-21T12:33:28.468 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-03-21T12:33:29.135 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - BaseOS 1.7 MB/s | 8.0 MB 00:04 2024-03-21T12:33:31.045 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-03-21T12:33:34.018 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.1 MB 00:01 2024-03-21T12:33:35.150 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - CRB 9.4 MB/s | 6.1 MB 00:00 2024-03-21T12:33:35.603 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 16 kB 00:00 2024-03-21T12:33:36.404 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 30 MB/s | 21 MB 00:00 2024-03-21T12:33:37.487 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 16 kB 00:01 2024-03-21T12:33:41.545 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-03-21T12:33:42.996 INFO:teuthology.orchestra.run.smithi155.stdout:Extra Packages for Enterprise Linux 3.8 MB/s | 21 MB 00:05 2024-03-21T12:33:43.744 INFO:teuthology.orchestra.run.smithi043.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-21T12:33:43.744 INFO:teuthology.orchestra.run.smithi043.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-21T12:33:43.831 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:33:43.840 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:33:43.840 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-03-21T12:33:43.840 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:33:43.840 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-03-21T12:33:43.840 INFO:teuthology.orchestra.run.smithi043.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-21T12:33:43.840 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-03-21T12:33:43.840 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-21T12:33:43.840 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-21T12:33:43.840 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-21T12:33:43.840 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:33:43.840 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-03-21T12:33:43.840 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:33:43.841 INFO:teuthology.orchestra.run.smithi043.stdout:Install 4 Packages 2024-03-21T12:33:43.841 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:33:43.841 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 76 M 2024-03-21T12:33:43.841 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 123 M 2024-03-21T12:33:43.841 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-03-21T12:33:44.055 INFO:teuthology.orchestra.run.smithi043.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 337 kB/s | 44 kB 00:00 2024-03-21T12:33:46.372 INFO:teuthology.orchestra.run.smithi043.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 6.1 MB/s | 15 MB 00:02 2024-03-21T12:33:47.921 INFO:teuthology.orchestra.run.smithi155.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-03-21T12:33:48.155 INFO:teuthology.orchestra.run.smithi043.stdout:(3/4): kernel-modules-core-5.14.0-430.el9.x86_6 6.7 MB/s | 27 MB 00:04 2024-03-21T12:33:48.714 INFO:teuthology.orchestra.run.smithi043.stdout:(4/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 7.0 MB/s | 33 MB 00:04 2024-03-21T12:33:48.714 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:33:48.714 INFO:teuthology.orchestra.run.smithi043.stdout:Total 16 MB/s | 76 MB 00:04 2024-03-21T12:33:49.411 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-03-21T12:33:49.842 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-03-21T12:33:49.842 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-03-21T12:33:50.036 INFO:teuthology.orchestra.run.smithi155.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-21T12:33:50.036 INFO:teuthology.orchestra.run.smithi155.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-21T12:33:50.130 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:33:50.134 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:33:50.134 INFO:teuthology.orchestra.run.smithi155.stdout: Package Architecture Version Repository Size 2024-03-21T12:33:50.134 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:33:50.134 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-03-21T12:33:50.134 INFO:teuthology.orchestra.run.smithi155.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-21T12:33:50.135 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-03-21T12:33:50.135 INFO:teuthology.orchestra.run.smithi155.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-21T12:33:50.135 INFO:teuthology.orchestra.run.smithi155.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-21T12:33:50.135 INFO:teuthology.orchestra.run.smithi155.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-21T12:33:50.135 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:33:50.135 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-03-21T12:33:50.135 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:33:50.135 INFO:teuthology.orchestra.run.smithi155.stdout:Install 4 Packages 2024-03-21T12:33:50.135 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:33:50.135 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 76 M 2024-03-21T12:33:50.136 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 123 M 2024-03-21T12:33:50.136 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-03-21T12:33:50.530 INFO:teuthology.orchestra.run.smithi155.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 485 kB/s | 44 kB 00:00 2024-03-21T12:33:50.885 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-03-21T12:33:50.885 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-03-21T12:33:50.889 INFO:teuthology.orchestra.run.smithi155.stdout:(2/4): kernel-core-5.14.0-430.el9.x86_64.rpm 33 MB/s | 15 MB 00:00 2024-03-21T12:33:51.289 INFO:teuthology.orchestra.run.smithi155.stdout:(3/4): kernel-modules-core-5.14.0-430.el9.x86_6 36 MB/s | 27 MB 00:00 2024-03-21T12:33:51.581 INFO:teuthology.orchestra.run.smithi155.stdout:(4/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 29 MB/s | 33 MB 00:01 2024-03-21T12:33:51.581 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:33:51.582 INFO:teuthology.orchestra.run.smithi155.stdout:Total 52 MB/s | 76 MB 00:01 2024-03-21T12:33:52.249 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-03-21T12:33:52.638 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-03-21T12:33:52.639 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-03-21T12:33:53.494 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-03-21T12:33:53.669 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-03-21T12:33:53.669 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-03-21T12:33:54.236 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-21T12:33:54.331 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-21T12:33:56.141 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-03-21T12:33:56.934 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-21T12:33:57.022 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-21T12:33:57.215 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-21T12:33:57.360 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-21T12:33:59.817 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-21T12:33:59.964 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-21T12:34:02.455 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-21T12:34:02.502 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-21T12:34:04.706 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-21T12:34:04.816 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-21T12:34:07.635 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-21T12:34:09.552 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-21T12:34:38.594 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-21T12:34:38.594 INFO:teuthology.orchestra.run.smithi043.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-21T12:34:38.594 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:34:38.639 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-21T12:34:38.875 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-21T12:34:38.875 INFO:teuthology.orchestra.run.smithi155.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-21T12:34:38.875 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:34:38.921 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-21T12:34:39.017 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-21T12:34:39.017 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-21T12:34:39.017 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-21T12:34:39.018 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-21T12:34:39.255 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-21T12:34:39.255 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-21T12:34:39.255 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-21T12:34:39.256 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-21T12:34:39.414 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-21T12:34:39.415 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:34:39.415 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-03-21T12:34:39.415 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-21T12:34:39.415 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-21T12:34:39.415 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-21T12:34:39.415 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-21T12:34:39.415 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:34:39.415 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:34:39.672 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-21T12:34:39.672 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:34:39.672 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-03-21T12:34:39.672 INFO:teuthology.orchestra.run.smithi155.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-21T12:34:39.672 INFO:teuthology.orchestra.run.smithi155.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-21T12:34:39.672 INFO:teuthology.orchestra.run.smithi155.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-21T12:34:39.673 INFO:teuthology.orchestra.run.smithi155.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-21T12:34:39.673 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:34:39.673 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:34:39.731 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q kernel | sort -rV | head -n 1 2024-03-21T12:34:39.834 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-21T12:34:39.835 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-21T12:34:39.835 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-21T12:34:39.835 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-21T12:34:39.835 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-21T12:34:39.835 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-21T12:34:39.835 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi043.front.sepia.ceph.com, path=None, version=distro) 2024-03-21T12:34:39.835 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q kernel | sort -rV | head -n 1 2024-03-21T12:34:39.918 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-21T12:34:39.918 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-21T12:34:39.918 DEBUG:teuthology.orchestra.run.smithi043:> sudo rpm -qi grub2-tools 2024-03-21T12:34:39.965 INFO:teuthology.orchestra.run.smithi043.stdout:Name : grub2-tools 2024-03-21T12:34:39.965 INFO:teuthology.orchestra.run.smithi043.stdout:Epoch : 1 2024-03-21T12:34:39.966 INFO:teuthology.orchestra.run.smithi043.stdout:Version : 2.06 2024-03-21T12:34:39.966 INFO:teuthology.orchestra.run.smithi043.stdout:Release : 68.el9 2024-03-21T12:34:39.966 INFO:teuthology.orchestra.run.smithi043.stdout:Architecture: x86_64 2024-03-21T12:34:39.966 INFO:teuthology.orchestra.run.smithi043.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-21T12:34:39.966 INFO:teuthology.orchestra.run.smithi043.stdout:Group : Unspecified 2024-03-21T12:34:39.966 INFO:teuthology.orchestra.run.smithi043.stdout:Size : 8264254 2024-03-21T12:34:39.966 INFO:teuthology.orchestra.run.smithi043.stdout:License : GPLv3+ 2024-03-21T12:34:39.966 INFO:teuthology.orchestra.run.smithi043.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-21T12:34:39.966 INFO:teuthology.orchestra.run.smithi043.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-21T12:34:39.966 INFO:teuthology.orchestra.run.smithi043.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-21T12:34:39.966 INFO:teuthology.orchestra.run.smithi043.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-21T12:34:39.966 INFO:teuthology.orchestra.run.smithi043.stdout:Packager : builder@centos.org 2024-03-21T12:34:39.967 INFO:teuthology.orchestra.run.smithi043.stdout:Vendor : CentOS 2024-03-21T12:34:39.967 INFO:teuthology.orchestra.run.smithi043.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-21T12:34:39.967 INFO:teuthology.orchestra.run.smithi043.stdout:Summary : Support tools for GRUB. 2024-03-21T12:34:39.967 INFO:teuthology.orchestra.run.smithi043.stdout:Description : 2024-03-21T12:34:39.967 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:34:39.967 INFO:teuthology.orchestra.run.smithi043.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-21T12:34:39.967 INFO:teuthology.orchestra.run.smithi043.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-21T12:34:39.967 INFO:teuthology.orchestra.run.smithi043.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-21T12:34:39.967 INFO:teuthology.orchestra.run.smithi043.stdout:hardware devices. 2024-03-21T12:34:39.967 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:34:39.967 INFO:teuthology.orchestra.run.smithi043.stdout:This subpackage provides tools for support of all platforms. 2024-03-21T12:34:39.968 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-21T12:34:39.968 INFO:teuthology.task.kernel:Updating grub on smithi043 to boot 5.14.0-430.el9.x86_64 2024-03-21T12:34:39.968 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-21T12:34:40.010 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q kernel | sort -rV | head -n 1 2024-03-21T12:34:40.119 INFO:teuthology.orchestra.run.smithi155.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-21T12:34:40.120 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-21T12:34:40.120 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-21T12:34:40.120 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-21T12:34:40.120 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-21T12:34:40.120 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-03-21T12:34:40.120 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi155.front.sepia.ceph.com, path=None, version=distro) 2024-03-21T12:34:40.120 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q kernel | sort -rV | head -n 1 2024-03-21T12:34:40.199 INFO:teuthology.orchestra.run.smithi155.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-21T12:34:40.200 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-21T12:34:40.200 DEBUG:teuthology.orchestra.run.smithi155:> sudo rpm -qi grub2-tools 2024-03-21T12:34:40.233 INFO:teuthology.orchestra.run.smithi155.stdout:Name : grub2-tools 2024-03-21T12:34:40.234 INFO:teuthology.orchestra.run.smithi155.stdout:Epoch : 1 2024-03-21T12:34:40.234 INFO:teuthology.orchestra.run.smithi155.stdout:Version : 2.06 2024-03-21T12:34:40.234 INFO:teuthology.orchestra.run.smithi155.stdout:Release : 68.el9 2024-03-21T12:34:40.234 INFO:teuthology.orchestra.run.smithi155.stdout:Architecture: x86_64 2024-03-21T12:34:40.234 INFO:teuthology.orchestra.run.smithi155.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-21T12:34:40.234 INFO:teuthology.orchestra.run.smithi155.stdout:Group : Unspecified 2024-03-21T12:34:40.234 INFO:teuthology.orchestra.run.smithi155.stdout:Size : 8264254 2024-03-21T12:34:40.234 INFO:teuthology.orchestra.run.smithi155.stdout:License : GPLv3+ 2024-03-21T12:34:40.234 INFO:teuthology.orchestra.run.smithi155.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-21T12:34:40.234 INFO:teuthology.orchestra.run.smithi155.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-21T12:34:40.234 INFO:teuthology.orchestra.run.smithi155.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-21T12:34:40.234 INFO:teuthology.orchestra.run.smithi155.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-21T12:34:40.234 INFO:teuthology.orchestra.run.smithi155.stdout:Packager : builder@centos.org 2024-03-21T12:34:40.235 INFO:teuthology.orchestra.run.smithi155.stdout:Vendor : CentOS 2024-03-21T12:34:40.235 INFO:teuthology.orchestra.run.smithi155.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-21T12:34:40.235 INFO:teuthology.orchestra.run.smithi155.stdout:Summary : Support tools for GRUB. 2024-03-21T12:34:40.235 INFO:teuthology.orchestra.run.smithi155.stdout:Description : 2024-03-21T12:34:40.235 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:34:40.235 INFO:teuthology.orchestra.run.smithi155.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-21T12:34:40.235 INFO:teuthology.orchestra.run.smithi155.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-21T12:34:40.235 INFO:teuthology.orchestra.run.smithi155.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-21T12:34:40.235 INFO:teuthology.orchestra.run.smithi155.stdout:hardware devices. 2024-03-21T12:34:40.235 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:34:40.235 INFO:teuthology.orchestra.run.smithi155.stdout:This subpackage provides tools for support of all platforms. 2024-03-21T12:34:40.236 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-21T12:34:40.236 INFO:teuthology.task.kernel:Updating grub on smithi155 to boot 5.14.0-430.el9.x86_64 2024-03-21T12:34:40.236 DEBUG:teuthology.orchestra.run.smithi155:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-21T12:34:40.459 INFO:teuthology.orchestra.run.smithi043.stderr:Generating grub configuration file ... 2024-03-21T12:34:40.779 INFO:teuthology.orchestra.run.smithi155.stderr:Generating grub configuration file ... 2024-03-21T12:34:41.889 INFO:teuthology.orchestra.run.smithi043.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-21T12:34:41.907 INFO:teuthology.orchestra.run.smithi043.stderr:done 2024-03-21T12:34:41.910 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2024-03-21T12:34:41.925 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.kgpLALfz6s 2024-03-21T12:34:41.926 DEBUG:teuthology.orchestra.run.smithi043:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.kgpLALfz6s 2024-03-21T12:34:41.998 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.kgpLALfz6s 2024-03-21T12:34:42.147 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.kgpLALfz6s is 6KB 2024-03-21T12:34:42.195 DEBUG:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.kgpLALfz6s 2024-03-21T12:34:42.210 DEBUG:teuthology.orchestra.run.smithi043:> sudo /bin/ls /boot/loader/entries || true 2024-03-21T12:34:42.233 INFO:teuthology.orchestra.run.smithi155.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-21T12:34:42.269 INFO:teuthology.orchestra.run.smithi155.stderr:done 2024-03-21T12:34:42.271 DEBUG:teuthology.orchestra.run.smithi155:> mktemp 2024-03-21T12:34:42.281 INFO:teuthology.orchestra.run.smithi043.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-21T12:34:42.281 INFO:teuthology.orchestra.run.smithi043.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-21T12:34:42.281 INFO:teuthology.orchestra.run.smithi043.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-21T12:34:42.281 INFO:teuthology.orchestra.run.smithi043.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-21T12:34:42.283 DEBUG:teuthology.orchestra.run.smithi043:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-21T12:34:42.287 INFO:teuthology.orchestra.run.smithi155.stdout:/tmp/tmp.8hSOjyrvHG 2024-03-21T12:34:42.287 DEBUG:teuthology.orchestra.run.smithi155:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.8hSOjyrvHG 2024-03-21T12:34:42.356 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0666 /tmp/tmp.8hSOjyrvHG 2024-03-21T12:34:42.459 DEBUG:teuthology.orchestra.run.smithi043:> sudo shutdown -r now 2024-03-21T12:34:42.499 DEBUG:teuthology.orchestra.remote:smithi155:/tmp/tmp.8hSOjyrvHG is 6KB 2024-03-21T12:34:42.547 DEBUG:teuthology.orchestra.run.smithi155:> rm -fr /tmp/tmp.8hSOjyrvHG 2024-03-21T12:34:42.562 DEBUG:teuthology.orchestra.run.smithi155:> sudo /bin/ls /boot/loader/entries || true 2024-03-21T12:34:42.627 INFO:teuthology.orchestra.run.smithi155.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-21T12:34:42.627 INFO:teuthology.orchestra.run.smithi155.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-21T12:34:42.627 INFO:teuthology.orchestra.run.smithi155.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-21T12:34:42.627 INFO:teuthology.orchestra.run.smithi155.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-21T12:34:42.628 DEBUG:teuthology.orchestra.run.smithi155:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-21T12:34:42.778 DEBUG:teuthology.orchestra.run.smithi155:> sudo shutdown -r now 2024-03-21T12:35:12.469 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-21T12:35:12.469 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-03-21T12:35:12.470 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T12:35:12.781 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-21T12:35:12.781 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-03-21T12:35:12.782 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T12:35:30.974 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.43 2024-03-21T12:35:31.230 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-03-21T12:35:39.982 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-03-21T12:35:39.983 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T12:35:40.231 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-03-21T12:35:40.232 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T12:35:43.038 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.43 2024-03-21T12:35:43.294 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-03-21T12:35:55.050 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-03-21T12:35:55.051 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T12:35:55.295 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-03-21T12:35:55.296 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T12:36:26.912 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-03-21T12:36:27.282 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-03-21T12:36:27.283 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-430.el9.x86_64"... 2024-03-21T12:36:27.283 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-03-21T12:36:27.298 INFO:teuthology.orchestra.run.smithi043.stdout:5.14.0-430.el9.x86_64 2024-03-21T12:36:27.298 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-21T12:36:27.298 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-21T12:36:27.298 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-21T12:36:28.300 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-21T12:36:28.300 DEBUG:teuthology.orchestra.run.smithi043:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-21T12:36:28.374 INFO:teuthology.orchestra.run.smithi043.stdout:ttyS1 2024-03-21T12:36:28.416 DEBUG:teuthology.parallel:result is None 2024-03-21T12:36:55.298 DEBUG:teuthology.orchestra.remote:timed out 2024-03-21T12:37:10.299 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-03-21T12:37:10.300 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T12:37:10.793 DEBUG:teuthology.orchestra.run.smithi155:> true 2024-03-21T12:37:11.179 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-03-21T12:37:11.179 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-430.el9.x86_64"... 2024-03-21T12:37:11.180 DEBUG:teuthology.orchestra.run.smithi155:> uname -r 2024-03-21T12:37:11.196 INFO:teuthology.orchestra.run.smithi155.stdout:5.14.0-430.el9.x86_64 2024-03-21T12:37:11.196 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-21T12:37:11.196 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-21T12:37:11.196 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-21T12:37:12.197 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-21T12:37:12.198 DEBUG:teuthology.orchestra.run.smithi155:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-21T12:37:12.271 INFO:teuthology.orchestra.run.smithi155.stdout:ttyS1 2024-03-21T12:37:12.309 DEBUG:teuthology.parallel:result is None 2024-03-21T12:37:12.309 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-21T12:37:12.317 INFO:teuthology.task.internal:Creating test directory... 2024-03-21T12:37:12.318 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-21T12:37:12.322 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-21T12:37:12.339 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-21T12:37:12.381 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-21T12:37:12.395 INFO:teuthology.task.internal:Creating archive directory... 2024-03-21T12:37:12.395 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-21T12:37:12.397 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-21T12:37:12.427 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-21T12:37:12.444 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-21T12:37:12.444 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-21T12:37:12.464 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-21T12:37:12.501 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T12:37:12.503 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T12:37:12.515 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T12:37:12.517 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T12:37:12.518 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-21T12:37:12.555 INFO:teuthology.task.internal:Configuring sudo... 2024-03-21T12:37:12.556 DEBUG:teuthology.orchestra.run.smithi043:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-21T12:37:12.559 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-21T12:37:12.588 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-21T12:37:12.606 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-21T12:37:12.606 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-21T12:37:12.629 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-21T12:37:12.644 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T12:37:12.730 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T12:37:12.778 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T12:37:12.861 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T12:37:12.904 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-21T12:37:12.904 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-21T12:37:12.973 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T12:37:13.018 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T12:37:13.071 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T12:37:13.154 DEBUG:teuthology.orchestra.run.smithi155:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T12:37:13.198 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-21T12:37:13.198 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-21T12:37:13.265 DEBUG:teuthology.orchestra.run.smithi043:> sudo service rsyslog restart 2024-03-21T12:37:13.268 DEBUG:teuthology.orchestra.run.smithi155:> sudo service rsyslog restart 2024-03-21T12:37:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T12:37:13.346 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T12:37:13.776 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-21T12:37:13.784 INFO:teuthology.task.internal:Starting timer... 2024-03-21T12:37:13.784 INFO:teuthology.run_tasks:Running task pcp... 2024-03-21T12:37:13.846 INFO:teuthology.run_tasks:Running task selinux... 2024-03-21T12:37:13.860 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-03-21T12:37:13.860 DEBUG:teuthology.orchestra.run.smithi043:> sudo service auditd rotate 2024-03-21T12:37:13.909 INFO:teuthology.orchestra.run.smithi043.stdout:Rotating logs: 2024-03-21T12:37:13.911 DEBUG:teuthology.orchestra.run.smithi155:> sudo service auditd rotate 2024-03-21T12:37:13.957 INFO:teuthology.orchestra.run.smithi155.stdout:Rotating logs: 2024-03-21T12:37:13.959 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-21T12:37:13.959 DEBUG:teuthology.orchestra.run.smithi043:> /usr/sbin/getenforce 2024-03-21T12:37:13.981 INFO:teuthology.orchestra.run.smithi043.stdout:Permissive 2024-03-21T12:37:13.981 DEBUG:teuthology.orchestra.run.smithi155:> /usr/sbin/getenforce 2024-03-21T12:37:14.019 INFO:teuthology.orchestra.run.smithi155.stdout:Permissive 2024-03-21T12:37:14.020 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi043.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi155.front.sepia.ceph.com': 'permissive'} 2024-03-21T12:37:14.020 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-03-21T12:37:14.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T12:37:14.051 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-03-21T12:37:14.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T12:37:14.088 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-21T12:37:14.089 DEBUG:teuthology.orchestra.run.smithi043:> sudo /usr/sbin/setenforce permissive 2024-03-21T12:37:14.124 DEBUG:teuthology.orchestra.run.smithi155:> sudo /usr/sbin/setenforce permissive 2024-03-21T12:37:14.162 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-21T12:37:14.175 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-21T12:37:14.194 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-21T12:37:14.195 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi043.front.sepia.ceph.com,smithi155.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-21T12:41:35.340 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi043.front.sepia.ceph.com'), Remote(name='ubuntu@smithi155.front.sepia.ceph.com')] 2024-03-21T12:41:35.342 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-03-21T12:41:35.343 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T12:41:35.435 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-03-21T12:41:35.521 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-03-21T12:41:35.521 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-03-21T12:41:35.522 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T12:41:35.614 DEBUG:teuthology.orchestra.run.smithi155:> true 2024-03-21T12:41:35.698 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-03-21T12:41:35.698 INFO:teuthology.run_tasks:Running task clock... 2024-03-21T12:41:35.709 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-21T12:41:35.710 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-21T12:41:35.710 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T12:41:35.712 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-21T12:41:35.713 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T12:41:35.743 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-21T12:41:35.760 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-21T12:41:35.770 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-21T12:41:35.787 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-21T12:41:35.790 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: ntpd: command not found 2024-03-21T12:41:35.804 INFO:teuthology.orchestra.run.smithi043.stdout:506 Cannot talk to daemon 2024-03-21T12:41:35.816 INFO:teuthology.orchestra.run.smithi155.stderr:sudo: ntpd: command not found 2024-03-21T12:41:35.822 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-21T12:41:35.830 INFO:teuthology.orchestra.run.smithi155.stdout:506 Cannot talk to daemon 2024-03-21T12:41:35.838 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-21T12:41:35.846 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-21T12:41:35.863 INFO:teuthology.orchestra.run.smithi155.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-21T12:41:35.893 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-03-21T12:41:35.897 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T12:41:35.898 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-03-21T12:41:35.898 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T12:41:35.898 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T12:41:35.898 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T12:41:35.898 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T12:41:35.914 INFO:teuthology.orchestra.run.smithi155.stderr:bash: line 1: ntpq: command not found 2024-03-21T12:41:35.918 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T12:41:35.918 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2024-03-21T12:41:35.918 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T12:41:35.918 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T12:41:35.918 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T12:41:35.918 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T12:41:35.920 INFO:teuthology.run_tasks:Running task pexec... 2024-03-21T12:41:35.930 INFO:teuthology.task.pexec:Executing custom commands... 2024-03-21T12:41:35.931 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi043.front.sepia.ceph.com 2024-03-21T12:41:35.932 DEBUG:teuthology.orchestra.run.smithi043:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-21T12:41:35.932 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi155.front.sepia.ceph.com 2024-03-21T12:41:35.932 DEBUG:teuthology.orchestra.run.smithi155:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-03-21T12:41:36.534 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:47 ago on Thu 21 Mar 2024 12:40:49 PM UTC. 2024-03-21T12:41:36.545 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:00:50 ago on Thu 21 Mar 2024 12:40:46 PM UTC. 2024-03-21T12:41:36.687 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:41:36.691 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:41:36.692 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:41:36.693 INFO:teuthology.orchestra.run.smithi155.stdout: Package Architecture Version Repository Size 2024-03-21T12:41:36.693 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:41:36.693 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-03-21T12:41:36.693 INFO:teuthology.orchestra.run.smithi155.stdout: runc x86_64 4:1.1.12-2.el9 appstream 3.1 M 2024-03-21T12:41:36.693 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:41:36.693 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-03-21T12:41:36.693 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:41:36.693 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-03-21T12:41:36.693 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:41:36.698 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:41:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-03-21T12:41:36.698 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:41:36.698 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-03-21T12:41:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: runc x86_64 4:1.1.12-2.el9 appstream 3.1 M 2024-03-21T12:41:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:41:36.698 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-03-21T12:41:36.698 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:41:36.698 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-03-21T12:41:36.698 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:41:36.702 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 3.1 M 2024-03-21T12:41:36.702 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 9.8 M 2024-03-21T12:41:36.702 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-03-21T12:41:36.705 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 3.1 M 2024-03-21T12:41:36.706 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 9.8 M 2024-03-21T12:41:36.706 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-03-21T12:41:41.322 INFO:teuthology.orchestra.run.smithi043.stdout:runc-1.1.12-2.el9.x86_64.rpm 715 kB/s | 3.1 MB 00:04 2024-03-21T12:41:41.322 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:41:41.322 INFO:teuthology.orchestra.run.smithi043.stdout:Total 693 kB/s | 3.1 MB 00:04 2024-03-21T12:41:41.362 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-03-21T12:41:41.369 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-03-21T12:41:41.369 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-03-21T12:41:41.387 INFO:teuthology.orchestra.run.smithi155.stdout:runc-1.1.12-2.el9.x86_64.rpm 693 kB/s | 3.1 MB 00:04 2024-03-21T12:41:41.387 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:41:41.388 INFO:teuthology.orchestra.run.smithi155.stdout:Total 684 kB/s | 3.1 MB 00:04 2024-03-21T12:41:41.426 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-03-21T12:41:41.432 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-03-21T12:41:41.432 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-03-21T12:41:41.454 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-03-21T12:41:41.455 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-03-21T12:41:41.514 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-03-21T12:41:41.515 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-03-21T12:41:41.696 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-03-21T12:41:41.763 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-03-21T12:41:41.786 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : runc-4:1.1.12-2.el9.x86_64 1/1 2024-03-21T12:41:41.867 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : runc-4:1.1.12-2.el9.x86_64 1/1 2024-03-21T12:41:42.349 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: runc-4:1.1.12-2.el9.x86_64 1/1 2024-03-21T12:41:42.361 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: runc-4:1.1.12-2.el9.x86_64 1/1 2024-03-21T12:41:42.655 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : runc-4:1.1.12-2.el9.x86_64 1/1 2024-03-21T12:41:42.655 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:41:42.655 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-03-21T12:41:42.655 INFO:teuthology.orchestra.run.smithi155.stdout: runc-4:1.1.12-2.el9.x86_64 2024-03-21T12:41:42.656 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:41:42.656 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:41:42.673 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : runc-4:1.1.12-2.el9.x86_64 1/1 2024-03-21T12:41:42.673 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:41:42.673 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-03-21T12:41:42.673 INFO:teuthology.orchestra.run.smithi043.stdout: runc-4:1.1.12-2.el9.x86_64 2024-03-21T12:41:42.674 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:41:42.674 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:41:42.889 DEBUG:teuthology.parallel:result is None 2024-03-21T12:41:42.917 DEBUG:teuthology.parallel:result is None 2024-03-21T12:41:42.918 INFO:teuthology.run_tasks:Running task install... 2024-03-21T12:41:42.927 DEBUG:teuthology.task.install:project ceph 2024-03-21T12:41:42.927 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '960fbbe4645c2becc8084669be194c95183c7753'}, 'extra_packages': ['cephadm']} 2024-03-21T12:41:42.927 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '960fbbe4645c2becc8084669be194c95183c7753'} 2024-03-21T12:41:42.927 INFO:teuthology.task.install:Using flavor: default 2024-03-21T12:41:42.933 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-21T12:41:42.933 INFO:teuthology.task.install:extra packages: [] 2024-03-21T12:41:42.933 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '960fbbe4645c2becc8084669be194c95183c7753', 'tag': None, 'wait_for_package': False} 2024-03-21T12:41:42.934 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=960fbbe4645c2becc8084669be194c95183c7753 2024-03-21T12:41:42.936 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '960fbbe4645c2becc8084669be194c95183c7753', 'tag': None, 'wait_for_package': False} 2024-03-21T12:41:42.936 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=960fbbe4645c2becc8084669be194c95183c7753 2024-03-21T12:41:43.082 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-guits-testing-2024-03-21-0818/960fbbe4645c2becc8084669be194c95183c7753/centos/9/flavors/default/ 2024-03-21T12:41:43.082 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2267.g960fbbe4 2024-03-21T12:41:43.087 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-guits-testing-2024-03-21-0818/960fbbe4645c2becc8084669be194c95183c7753/centos/9/flavors/default/ 2024-03-21T12:41:43.087 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2267.g960fbbe4 2024-03-21T12:41:43.197 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-testing-2024-03-21-0818/960fbbe4645c2becc8084669be194c95183c7753/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-testing-2024-03-21-0818/960fbbe4645c2becc8084669be194c95183c7753/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-testing-2024-03-21-0818/960fbbe4645c2becc8084669be194c95183c7753/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-21T12:41:43.197 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-21T12:41:43.197 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-21T12:41:43.204 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-testing-2024-03-21-0818/960fbbe4645c2becc8084669be194c95183c7753/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-testing-2024-03-21-0818/960fbbe4645c2becc8084669be194c95183c7753/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-guits-testing-2024-03-21-0818/960fbbe4645c2becc8084669be194c95183c7753/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-21T12:41:43.205 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-21T12:41:43.205 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-21T12:41:43.224 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-21T12:41:43.225 DEBUG:teuthology.orchestra.run.smithi043:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/960fbbe4645c2becc8084669be194c95183c7753/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-21T12:41:43.232 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-03-21T12:41:43.232 DEBUG:teuthology.orchestra.run.smithi155:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/960fbbe4645c2becc8084669be194c95183c7753/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-21T12:41:43.292 DEBUG:teuthology.orchestra.run.smithi043:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-21T12:41:43.301 DEBUG:teuthology.orchestra.run.smithi155:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-21T12:41:43.374 DEBUG:teuthology.orchestra.run.smithi043:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-21T12:41:43.381 DEBUG:teuthology.orchestra.run.smithi155:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-21T12:41:43.442 INFO:teuthology.orchestra.run.smithi043.stdout:check_obsoletes = 1 2024-03-21T12:41:43.443 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum clean all 2024-03-21T12:41:43.448 INFO:teuthology.orchestra.run.smithi155.stdout:check_obsoletes = 1 2024-03-21T12:41:43.450 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum clean all 2024-03-21T12:41:43.688 INFO:teuthology.orchestra.run.smithi043.stdout:74 files removed 2024-03-21T12:41:43.704 INFO:teuthology.orchestra.run.smithi155.stdout:74 files removed 2024-03-21T12:41:43.715 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-radosgw 2024-03-21T12:41:43.736 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-radosgw 2024-03-21T12:41:44.519 INFO:teuthology.orchestra.run.smithi043.stdout:ceph packages for x86_64 169 kB/s | 83 kB 00:00 2024-03-21T12:41:44.528 INFO:teuthology.orchestra.run.smithi155.stdout:ceph packages for x86_64 174 kB/s | 83 kB 00:00 2024-03-21T12:41:44.904 INFO:teuthology.orchestra.run.smithi155.stdout:ceph noarch packages 36 kB/s | 12 kB 00:00 2024-03-21T12:41:44.953 INFO:teuthology.orchestra.run.smithi043.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-03-21T12:41:45.221 INFO:teuthology.orchestra.run.smithi155.stdout:ceph source packages 6.8 kB/s | 1.9 kB 00:00 2024-03-21T12:41:45.278 INFO:teuthology.orchestra.run.smithi043.stdout:ceph source packages 6.7 kB/s | 1.9 kB 00:00 2024-03-21T12:41:45.723 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-21T12:41:46.081 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS-9 - AppStream 8.8 MB/s | 6.7 MB 00:00 2024-03-21T12:41:47.107 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS-9 - BaseOS 7.6 MB/s | 2.0 MB 00:00 2024-03-21T12:41:47.539 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS-9 - BaseOS 7.4 MB/s | 2.0 MB 00:00 2024-03-21T12:41:48.039 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-03-21T12:41:49.791 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-03-21T12:41:51.513 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 2.2 MB/s | 8.0 MB 00:03 2024-03-21T12:41:54.433 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - CRB 5.2 MB/s | 6.1 MB 00:01 2024-03-21T12:41:56.117 INFO:teuthology.orchestra.run.smithi155.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 16 kB 00:00 2024-03-21T12:41:59.252 INFO:teuthology.orchestra.run.smithi155.stdout:Extra Packages for Enterprise Linux 6.9 MB/s | 21 MB 00:03 2024-03-21T12:42:04.202 INFO:teuthology.orchestra.run.smithi155.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-03-21T12:42:06.663 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:42:06.665 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:42:06.666 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-03-21T12:42:06.666 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:42:06.666 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-03-21T12:42:06.666 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 9.9 M 2024-03-21T12:42:06.666 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrading: 2024-03-21T12:42:06.666 INFO:teuthology.orchestra.run.smithi155.stdout: librados2 x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 3.4 M 2024-03-21T12:42:06.666 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1 x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 3.1 M 2024-03-21T12:42:06.666 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-03-21T12:42:06.666 INFO:teuthology.orchestra.run.smithi155.stdout: boost-program-options 2024-03-21T12:42:06.666 INFO:teuthology.orchestra.run.smithi155.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-21T12:42:06.666 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 5.4 M 2024-03-21T12:42:06.667 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 20 M 2024-03-21T12:42:06.667 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 25 k 2024-03-21T12:42:06.667 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-21T12:42:06.667 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-21T12:42:06.667 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-21T12:42:06.667 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2 x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 746 k 2024-03-21T12:42:06.667 INFO:teuthology.orchestra.run.smithi155.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-21T12:42:06.667 INFO:teuthology.orchestra.run.smithi155.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-21T12:42:06.667 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-21T12:42:06.667 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1 x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 501 k 2024-03-21T12:42:06.667 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-21T12:42:06.667 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2 x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 5.1 M 2024-03-21T12:42:06.668 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-21T12:42:06.668 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-21T12:42:06.668 INFO:teuthology.orchestra.run.smithi155.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-21T12:42:06.668 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse 2024-03-21T12:42:06.668 INFO:teuthology.orchestra.run.smithi155.stdout: x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 45 k 2024-03-21T12:42:06.668 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 131 k 2024-03-21T12:42:06.668 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 162 k 2024-03-21T12:42:06.668 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-21T12:42:06.668 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 320 k 2024-03-21T12:42:06.668 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 298 k 2024-03-21T12:42:06.668 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 99 k 2024-03-21T12:42:06.668 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-21T12:42:06.668 INFO:teuthology.orchestra.run.smithi155.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-21T12:42:06.668 INFO:teuthology.orchestra.run.smithi155.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-21T12:42:06.668 INFO:teuthology.orchestra.run.smithi155.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-21T12:42:06.669 INFO:teuthology.orchestra.run.smithi155.stdout:Installing weak dependencies: 2024-03-21T12:42:06.669 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-21T12:42:06.669 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:42:06.669 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-03-21T12:42:06.669 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:42:06.669 INFO:teuthology.orchestra.run.smithi155.stdout:Install 30 Packages 2024-03-21T12:42:06.669 INFO:teuthology.orchestra.run.smithi155.stdout:Upgrade 2 Packages 2024-03-21T12:42:06.669 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:42:06.669 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 59 M 2024-03-21T12:42:06.670 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-03-21T12:42:06.696 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:14 2024-03-21T12:42:07.636 INFO:teuthology.orchestra.run.smithi155.stdout:(1/32): ceph-base-19.0.0-2267.g960fbbe4.el9.x86 7.2 MB/s | 5.4 MB 00:00 2024-03-21T12:42:07.695 INFO:teuthology.orchestra.run.smithi155.stdout:(2/32): ceph-selinux-19.0.0-2267.g960fbbe4.el9. 432 kB/s | 25 kB 00:00 2024-03-21T12:42:07.828 INFO:teuthology.orchestra.run.smithi155.stdout:(3/32): libcephfs2-19.0.0-2267.g960fbbe4.el9.x8 5.5 MB/s | 746 kB 00:00 2024-03-21T12:42:07.912 INFO:teuthology.orchestra.run.smithi155.stdout:(4/32): libradosstriper1-19.0.0-2267.g960fbbe4. 5.9 MB/s | 501 kB 00:00 2024-03-21T12:42:08.204 INFO:teuthology.orchestra.run.smithi155.stdout:(5/32): ceph-radosgw-19.0.0-2267.g960fbbe4.el9. 7.5 MB/s | 9.9 MB 00:01 2024-03-21T12:42:08.262 INFO:teuthology.orchestra.run.smithi155.stdout:(6/32): python3-ceph-argparse-19.0.0-2267.g960f 776 kB/s | 45 kB 00:00 2024-03-21T12:42:08.354 INFO:teuthology.orchestra.run.smithi155.stdout:(7/32): librgw2-19.0.0-2267.g960fbbe4.el9.x86_6 12 MB/s | 5.1 MB 00:00 2024-03-21T12:42:08.371 INFO:teuthology.orchestra.run.smithi155.stdout:(8/32): python3-ceph-common-19.0.0-2267.g960fbb 1.2 MB/s | 131 kB 00:00 2024-03-21T12:42:08.413 INFO:teuthology.orchestra.run.smithi155.stdout:(9/32): python3-cephfs-19.0.0-2267.g960fbbe4.el 2.7 MB/s | 162 kB 00:00 2024-03-21T12:42:08.438 INFO:teuthology.orchestra.run.smithi155.stdout:(10/32): python3-rados-19.0.0-2267.g960fbbe4.el 4.7 MB/s | 320 kB 00:00 2024-03-21T12:42:08.472 INFO:teuthology.orchestra.run.smithi155.stdout:(11/32): python3-rbd-19.0.0-2267.g960fbbe4.el9. 5.0 MB/s | 298 kB 00:00 2024-03-21T12:42:08.497 INFO:teuthology.orchestra.run.smithi155.stdout:(12/32): python3-rgw-19.0.0-2267.g960fbbe4.el9. 1.7 MB/s | 99 kB 00:00 2024-03-21T12:42:08.614 INFO:teuthology.orchestra.run.smithi155.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 746 kB/s | 106 kB 00:00 2024-03-21T12:42:08.639 INFO:teuthology.orchestra.run.smithi155.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 159 kB 00:00 2024-03-21T12:42:08.656 INFO:teuthology.orchestra.run.smithi155.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-21T12:42:08.706 INFO:teuthology.orchestra.run.smithi155.stdout:(16/32): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 661 kB 00:00 2024-03-21T12:42:08.732 INFO:teuthology.orchestra.run.smithi155.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-21T12:42:08.757 INFO:teuthology.orchestra.run.smithi155.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 825 kB/s | 41 kB 00:00 2024-03-21T12:42:09.024 INFO:teuthology.orchestra.run.smithi155.stdout:(19/32): ceph-common-19.0.0-2267.g960fbbe4.el9. 9.6 MB/s | 20 MB 00:02 2024-03-21T12:42:09.207 INFO:teuthology.orchestra.run.smithi155.stdout:(20/32): qatlib-service-23.11.0-1.el9.x86_64.rp 77 kB/s | 35 kB 00:00 2024-03-21T12:42:09.374 INFO:teuthology.orchestra.run.smithi155.stdout:(21/32): qatzip-libs-1.1.2-2.el9.x86_64.rpm 127 kB/s | 44 kB 00:00 2024-03-21T12:42:09.441 INFO:teuthology.orchestra.run.smithi155.stdout:(22/32): qatlib-23.11.0-1.el9.x86_64.rpm 305 kB/s | 216 kB 00:00 2024-03-21T12:42:09.633 INFO:teuthology.orchestra.run.smithi155.stdout:(23/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 726 kB/s | 309 kB 00:00 2024-03-21T12:42:09.717 INFO:teuthology.orchestra.run.smithi155.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 583 kB/s | 49 kB 00:00 2024-03-21T12:42:09.742 INFO:teuthology.orchestra.run.smithi155.stdout:(25/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 94 kB/s | 26 kB 00:00 2024-03-21T12:42:09.801 INFO:teuthology.orchestra.run.smithi155.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 805 kB/s | 67 kB 00:00 2024-03-21T12:42:09.893 INFO:teuthology.orchestra.run.smithi155.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-03-21T12:42:10.076 INFO:teuthology.orchestra.run.smithi155.stdout:(28/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.5 MB/s | 838 kB 00:00 2024-03-21T12:42:10.143 INFO:teuthology.orchestra.run.smithi155.stdout:(29/32): libarrow-9.0.0-10.el9.x86_64.rpm 5.7 MB/s | 4.4 MB 00:00 2024-03-21T12:42:10.177 INFO:teuthology.orchestra.run.smithi155.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 5.6 MB/s | 1.6 MB 00:00 2024-03-21T12:42:10.535 INFO:teuthology.orchestra.run.smithi155.stdout:(31/32): librados2-19.0.0-2267.g960fbbe4.el9.x8 7.4 MB/s | 3.4 MB 00:00 2024-03-21T12:42:10.610 INFO:teuthology.orchestra.run.smithi155.stdout:(32/32): librbd1-19.0.0-2267.g960fbbe4.el9.x86_ 6.7 MB/s | 3.1 MB 00:00 2024-03-21T12:42:10.617 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:42:10.617 INFO:teuthology.orchestra.run.smithi155.stdout:Total 15 MB/s | 59 MB 00:03 2024-03-21T12:42:10.650 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-03-21T12:42:10.690 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-03-21T12:42:10.690 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-03-21T12:42:11.192 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-03-21T12:42:11.192 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-03-21T12:42:12.613 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-03-21T12:42:12.697 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-03-21T12:42:12.740 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-03-21T12:42:12.765 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-03-21T12:42:12.788 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-03-21T12:42:12.805 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-03-21T12:42:13.030 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 5/34 2024-03-21T12:42:13.085 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 5/34 2024-03-21T12:42:13.408 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-03-21T12:42:13.475 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librados2-2:19.0.0-2267.g960fbbe4.el9.x86_64 7/34 2024-03-21T12:42:13.550 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:19.0.0-2267.g960fbbe4.el9.x86_64 7/34 2024-03-21T12:42:13.637 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rados-2:19.0.0-2267.g960fbbe4.el9.x86_64 8/34 2024-03-21T12:42:13.677 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs2-2:19.0.0-2267.g960fbbe4.el9.x86_64 9/34 2024-03-21T12:42:13.756 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephfs2-2:19.0.0-2267.g960fbbe4.el9.x86_64 9/34 2024-03-21T12:42:13.785 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-03-21T12:42:13.820 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-argparse-2:19.0.0-2267.g960fbbe4.el9. 11/34 2024-03-21T12:42:13.888 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cephfs-2:19.0.0-2267.g960fbbe4.el9.x86_64 12/34 2024-03-21T12:42:13.913 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libradosstriper1-2:19.0.0-2267.g960fbbe4.el9.x86_6 13/34 2024-03-21T12:42:13.958 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2267.g960fbbe4.el9.x86_6 13/34 2024-03-21T12:42:13.986 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 14/34 2024-03-21T12:42:14.093 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-03-21T12:42:14.137 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-03-21T12:42:14.178 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-03-21T12:42:14.210 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-03-21T12:42:14.239 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-03-21T12:42:14.267 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-03-21T12:42:14.577 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-03-21T12:42:14.602 INFO:teuthology.orchestra.run.smithi155.stdout: Upgrading : librbd1-2:19.0.0-2267.g960fbbe4.el9.x86_64 22/34 2024-03-21T12:42:14.691 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:19.0.0-2267.g960fbbe4.el9.x86_64 22/34 2024-03-21T12:42:14.723 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 23/34 2024-03-21T12:42:14.768 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-03-21T12:42:15.226 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-03-21T12:42:15.254 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librgw2-2:19.0.0-2267.g960fbbe4.el9.x86_64 26/34 2024-03-21T12:42:15.308 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librgw2-2:19.0.0-2267.g960fbbe4.el9.x86_64 26/34 2024-03-21T12:42:15.377 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 27/34 2024-03-21T12:42:15.406 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ceph-common-2:19.0.0-2267.g960fbbe4.el9.x8 28/34 2024-03-21T12:42:17.607 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:19.0.0-2267.g960fbbe4.el9.x86_64 29/34 2024-03-21T12:42:17.661 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-common-2:19.0.0-2267.g960fbbe4.el9.x86_64 29/34 2024-03-21T12:42:18.220 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-common-2:19.0.0-2267.g960fbbe4.el9.x86_64 29/34 2024-03-21T12:42:18.256 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-base-2:19.0.0-2267.g960fbbe4.el9.x86_64 30/34 2024-03-21T12:42:18.330 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-base-2:19.0.0-2267.g960fbbe4.el9.x86_64 30/34 2024-03-21T12:42:18.331 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-21T12:42:18.331 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-21T12:42:18.331 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:42:18.374 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-selinux-2:19.0.0-2267.g960fbbe4.el9.x86_64 31/34 2024-03-21T12:42:18.495 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 773 kB/s | 6.1 MB 00:08 2024-03-21T12:42:20.030 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 51 kB/s | 16 kB 00:00 2024-03-21T12:42:21.956 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-03-21T12:42:27.206 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-03-21T12:42:28.123 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2267.g960fbbe4.el9.x86_64 31/34 2024-03-21T12:42:28.123 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /sys 2024-03-21T12:42:28.123 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /proc 2024-03-21T12:42:28.123 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /mnt 2024-03-21T12:42:28.123 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /var/tmp 2024-03-21T12:42:28.123 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /home 2024-03-21T12:42:28.123 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /root 2024-03-21T12:42:28.123 INFO:teuthology.orchestra.run.smithi155.stdout:skipping the directory /tmp 2024-03-21T12:42:28.123 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:42:29.077 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-radosgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 32/34 2024-03-21T12:42:29.101 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 32/34 2024-03-21T12:42:29.101 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T12:42:29.101 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-21T12:42:29.102 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T12:42:29.102 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T12:42:29.102 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:42:29.103 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-21T12:42:29.179 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-21T12:42:29.179 INFO:teuthology.orchestra.run.smithi155.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-21T12:42:29.710 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:42:29.712 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:42:29.712 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-03-21T12:42:29.713 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:42:29.713 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-03-21T12:42:29.713 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 9.9 M 2024-03-21T12:42:29.713 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrading: 2024-03-21T12:42:29.713 INFO:teuthology.orchestra.run.smithi043.stdout: librados2 x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 3.4 M 2024-03-21T12:42:29.713 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1 x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 3.1 M 2024-03-21T12:42:29.713 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-03-21T12:42:29.713 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options 2024-03-21T12:42:29.713 INFO:teuthology.orchestra.run.smithi043.stdout: x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-21T12:42:29.713 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 5.4 M 2024-03-21T12:42:29.713 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 20 M 2024-03-21T12:42:29.713 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 25 k 2024-03-21T12:42:29.713 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-21T12:42:29.713 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-21T12:42:29.714 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-21T12:42:29.714 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2 x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 746 k 2024-03-21T12:42:29.714 INFO:teuthology.orchestra.run.smithi043.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-21T12:42:29.714 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-21T12:42:29.714 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-21T12:42:29.714 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1 x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 501 k 2024-03-21T12:42:29.714 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-21T12:42:29.714 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2 x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 5.1 M 2024-03-21T12:42:29.714 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-21T12:42:29.714 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-21T12:42:29.715 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-21T12:42:29.715 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse 2024-03-21T12:42:29.715 INFO:teuthology.orchestra.run.smithi043.stdout: x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 45 k 2024-03-21T12:42:29.715 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 131 k 2024-03-21T12:42:29.715 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 162 k 2024-03-21T12:42:29.715 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-21T12:42:29.715 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 320 k 2024-03-21T12:42:29.715 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 298 k 2024-03-21T12:42:29.715 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 99 k 2024-03-21T12:42:29.715 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-21T12:42:29.715 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-21T12:42:29.715 INFO:teuthology.orchestra.run.smithi043.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-21T12:42:29.715 INFO:teuthology.orchestra.run.smithi043.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-21T12:42:29.716 INFO:teuthology.orchestra.run.smithi043.stdout:Installing weak dependencies: 2024-03-21T12:42:29.716 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-21T12:42:29.716 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:42:29.716 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-03-21T12:42:29.716 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:42:29.716 INFO:teuthology.orchestra.run.smithi043.stdout:Install 30 Packages 2024-03-21T12:42:29.716 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrade 2 Packages 2024-03-21T12:42:29.716 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:42:29.716 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 59 M 2024-03-21T12:42:29.717 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-03-21T12:42:30.342 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-21T12:42:30.343 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-base-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/34 2024-03-21T12:42:30.343 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-common-2:19.0.0-2267.g960fbbe4.el9.x86_64 2/34 2024-03-21T12:42:30.343 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-radosgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 3/34 2024-03-21T12:42:30.343 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-selinux-2:19.0.0-2267.g960fbbe4.el9.x86_64 4/34 2024-03-21T12:42:30.343 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs2-2:19.0.0-2267.g960fbbe4.el9.x86_64 5/34 2024-03-21T12:42:30.343 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libradosstriper1-2:19.0.0-2267.g960fbbe4.el9.x86_6 6/34 2024-03-21T12:42:30.343 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librgw2-2:19.0.0-2267.g960fbbe4.el9.x86_64 7/34 2024-03-21T12:42:30.343 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2267.g960fbbe4.el9. 8/34 2024-03-21T12:42:30.343 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ceph-common-2:19.0.0-2267.g960fbbe4.el9.x8 9/34 2024-03-21T12:42:30.343 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cephfs-2:19.0.0-2267.g960fbbe4.el9.x86_64 10/34 2024-03-21T12:42:30.344 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rados-2:19.0.0-2267.g960fbbe4.el9.x86_64 11/34 2024-03-21T12:42:30.344 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 12/34 2024-03-21T12:42:30.344 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 13/34 2024-03-21T12:42:30.344 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-03-21T12:42:30.344 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-03-21T12:42:30.344 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-03-21T12:42:30.344 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-03-21T12:42:30.344 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-03-21T12:42:30.344 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-03-21T12:42:30.344 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 20/34 2024-03-21T12:42:30.344 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 21/34 2024-03-21T12:42:30.344 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 22/34 2024-03-21T12:42:30.344 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-03-21T12:42:30.344 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-03-21T12:42:30.345 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-03-21T12:42:30.345 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-03-21T12:42:30.345 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-03-21T12:42:30.345 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-03-21T12:42:30.345 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-03-21T12:42:30.345 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-03-21T12:42:30.345 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:19.0.0-2267.g960fbbe4.el9.x86_64 31/34 2024-03-21T12:42:30.346 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-03-21T12:42:30.346 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:19.0.0-2267.g960fbbe4.el9.x86_64 33/34 2024-03-21T12:42:31.240 INFO:teuthology.orchestra.run.smithi043.stdout:(1/32): ceph-base-19.0.0-2267.g960fbbe4.el9.x86 5.3 MB/s | 5.4 MB 00:01 2024-03-21T12:42:31.324 INFO:teuthology.orchestra.run.smithi043.stdout:(2/32): ceph-selinux-19.0.0-2267.g960fbbe4.el9. 302 kB/s | 25 kB 00:00 2024-03-21T12:42:31.412 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-03-21T12:42:31.412 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:42:31.412 INFO:teuthology.orchestra.run.smithi155.stdout:Upgraded: 2024-03-21T12:42:31.412 INFO:teuthology.orchestra.run.smithi155.stdout: librados2-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:31.412 INFO:teuthology.orchestra.run.smithi155.stdout: librbd1-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:31.412 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-03-21T12:42:31.413 INFO:teuthology.orchestra.run.smithi155.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-21T12:42:31.413 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:31.413 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-common-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:31.413 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-radosgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:31.413 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-selinux-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:31.413 INFO:teuthology.orchestra.run.smithi155.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-21T12:42:31.413 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-21T12:42:31.413 INFO:teuthology.orchestra.run.smithi155.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-21T12:42:31.413 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:31.413 INFO:teuthology.orchestra.run.smithi155.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-21T12:42:31.413 INFO:teuthology.orchestra.run.smithi155.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-21T12:42:31.413 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-21T12:42:31.413 INFO:teuthology.orchestra.run.smithi155.stdout: libradosstriper1-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:31.413 INFO:teuthology.orchestra.run.smithi155.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-21T12:42:31.414 INFO:teuthology.orchestra.run.smithi155.stdout: librgw2-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:31.414 INFO:teuthology.orchestra.run.smithi155.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-21T12:42:31.414 INFO:teuthology.orchestra.run.smithi155.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-21T12:42:31.414 INFO:teuthology.orchestra.run.smithi155.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-21T12:42:31.414 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:31.414 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-common-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:31.414 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cephfs-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:31.414 INFO:teuthology.orchestra.run.smithi155.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-21T12:42:31.414 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:31.414 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:31.414 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:31.415 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-21T12:42:31.415 INFO:teuthology.orchestra.run.smithi155.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-21T12:42:31.415 INFO:teuthology.orchestra.run.smithi155.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-21T12:42:31.415 INFO:teuthology.orchestra.run.smithi155.stdout: re2-1:20211101-3.el9.x86_64 2024-03-21T12:42:31.415 INFO:teuthology.orchestra.run.smithi155.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-21T12:42:31.415 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:42:31.415 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:42:31.432 INFO:teuthology.orchestra.run.smithi043.stdout:(3/32): libcephfs2-19.0.0-2267.g960fbbe4.el9.x8 6.7 MB/s | 746 kB 00:00 2024-03-21T12:42:31.541 INFO:teuthology.orchestra.run.smithi043.stdout:(4/32): libradosstriper1-19.0.0-2267.g960fbbe4. 4.5 MB/s | 501 kB 00:00 2024-03-21T12:42:31.783 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-test 2024-03-21T12:42:32.133 INFO:teuthology.orchestra.run.smithi043.stdout:(5/32): librgw2-19.0.0-2267.g960fbbe4.el9.x86_6 8.7 MB/s | 5.1 MB 00:00 2024-03-21T12:42:32.333 INFO:teuthology.orchestra.run.smithi043.stdout:(6/32): ceph-common-19.0.0-2267.g960fbbe4.el9.x 9.8 MB/s | 20 MB 00:02 2024-03-21T12:42:32.342 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:00:28 ago on Thu 21 Mar 2024 12:42:04 PM UTC. 2024-03-21T12:42:32.350 INFO:teuthology.orchestra.run.smithi043.stdout:(7/32): python3-ceph-argparse-19.0.0-2267.g960f 208 kB/s | 45 kB 00:00 2024-03-21T12:42:32.434 INFO:teuthology.orchestra.run.smithi043.stdout:(8/32): python3-ceph-common-19.0.0-2267.g960fbb 1.3 MB/s | 131 kB 00:00 2024-03-21T12:42:32.459 INFO:teuthology.orchestra.run.smithi043.stdout:(9/32): python3-cephfs-19.0.0-2267.g960fbbe4.el 1.5 MB/s | 162 kB 00:00 2024-03-21T12:42:32.499 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:42:32.499 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:42:32.499 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-03-21T12:42:32.499 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:42:32.499 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-03-21T12:42:32.500 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 48 M 2024-03-21T12:42:32.500 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-03-21T12:42:32.500 INFO:teuthology.orchestra.run.smithi155.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-21T12:42:32.500 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 163 k 2024-03-21T12:42:32.500 INFO:teuthology.orchestra.run.smithi155.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-21T12:42:32.500 INFO:teuthology.orchestra.run.smithi155.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-21T12:42:32.500 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-21T12:42:32.500 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:42:32.500 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-03-21T12:42:32.500 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:42:32.500 INFO:teuthology.orchestra.run.smithi155.stdout:Install 6 Packages 2024-03-21T12:42:32.500 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:42:32.501 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 49 M 2024-03-21T12:42:32.501 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 206 M 2024-03-21T12:42:32.501 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-03-21T12:42:32.526 INFO:teuthology.orchestra.run.smithi043.stdout:(10/32): python3-rados-19.0.0-2267.g960fbbe4.el 3.4 MB/s | 320 kB 00:00 2024-03-21T12:42:32.551 INFO:teuthology.orchestra.run.smithi043.stdout:(11/32): python3-rbd-19.0.0-2267.g960fbbe4.el9. 3.2 MB/s | 298 kB 00:00 2024-03-21T12:42:32.635 INFO:teuthology.orchestra.run.smithi043.stdout:(12/32): python3-rgw-19.0.0-2267.g960fbbe4.el9. 917 kB/s | 99 kB 00:00 2024-03-21T12:42:32.843 INFO:teuthology.orchestra.run.smithi043.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 363 kB/s | 106 kB 00:00 2024-03-21T12:42:32.894 INFO:teuthology.orchestra.run.smithi043.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 615 kB/s | 159 kB 00:00 2024-03-21T12:42:32.919 INFO:teuthology.orchestra.run.smithi043.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 587 kB/s | 44 kB 00:00 2024-03-21T12:42:32.924 INFO:teuthology.orchestra.run.smithi155.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 873 kB/s | 217 kB 00:00 2024-03-21T12:42:33.058 INFO:teuthology.orchestra.run.smithi155.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 2.2 MB/s | 300 kB 00:00 2024-03-21T12:42:33.124 INFO:teuthology.orchestra.run.smithi155.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 942 kB/s | 63 kB 00:00 2024-03-21T12:42:33.150 INFO:teuthology.orchestra.run.smithi155.stdout:(4/6): libcephsqlite-19.0.0-2267.g960fbbe4.el9. 344 kB/s | 163 kB 00:00 2024-03-21T12:42:33.194 INFO:teuthology.orchestra.run.smithi043.stdout:(16/32): librdkafka-1.6.1-102.el9.x86_64.rpm 2.2 MB/s | 661 kB 00:00 2024-03-21T12:42:33.219 INFO:teuthology.orchestra.run.smithi043.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 940 kB/s | 282 kB 00:00 2024-03-21T12:42:33.253 INFO:teuthology.orchestra.run.smithi043.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 708 kB/s | 41 kB 00:00 2024-03-21T12:42:33.529 INFO:teuthology.orchestra.run.smithi043.stdout:(19/32): ceph-radosgw-19.0.0-2267.g960fbbe4.el9 3.0 MB/s | 9.9 MB 00:03 2024-03-21T12:42:33.579 INFO:teuthology.orchestra.run.smithi043.stdout:(20/32): qatlib-service-23.11.0-1.el9.x86_64.rp 107 kB/s | 35 kB 00:00 2024-03-21T12:42:33.679 INFO:teuthology.orchestra.run.smithi043.stdout:(21/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 3.0 MB/s | 309 kB 00:00 2024-03-21T12:42:33.829 INFO:teuthology.orchestra.run.smithi043.stdout:(22/32): libarrow-9.0.0-10.el9.x86_64.rpm 29 MB/s | 4.4 MB 00:00 2024-03-21T12:42:33.863 INFO:teuthology.orchestra.run.smithi043.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 786 kB/s | 26 kB 00:00 2024-03-21T12:42:33.888 INFO:teuthology.orchestra.run.smithi043.stdout:(24/32): qatzip-libs-1.1.2-2.el9.x86_64.rpm 124 kB/s | 44 kB 00:00 2024-03-21T12:42:33.913 INFO:teuthology.orchestra.run.smithi043.stdout:(25/32): qatlib-23.11.0-1.el9.x86_64.rpm 312 kB/s | 216 kB 00:00 2024-03-21T12:42:33.939 INFO:teuthology.orchestra.run.smithi043.stdout:(26/32): liboath-2.6.7-2.el9.x86_64.rpm 643 kB/s | 49 kB 00:00 2024-03-21T12:42:33.972 INFO:teuthology.orchestra.run.smithi043.stdout:(27/32): libunwind-1.6.2-1.el9.x86_64.rpm 806 kB/s | 67 kB 00:00 2024-03-21T12:42:33.989 INFO:teuthology.orchestra.run.smithi043.stdout:(28/32): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-03-21T12:42:34.064 INFO:teuthology.orchestra.run.smithi043.stdout:(29/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.4 MB/s | 838 kB 00:00 2024-03-21T12:42:34.115 INFO:teuthology.orchestra.run.smithi043.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-03-21T12:42:34.124 INFO:teuthology.orchestra.run.smithi155.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 187 kB/s | 187 kB 00:01 2024-03-21T12:42:34.848 INFO:teuthology.orchestra.run.smithi043.stdout:(31/32): librados2-19.0.0-2267.g960fbbe4.el9.x8 4.0 MB/s | 3.4 MB 00:00 2024-03-21T12:42:35.782 INFO:teuthology.orchestra.run.smithi043.stdout:(32/32): librbd1-19.0.0-2267.g960fbbe4.el9.x86_ 1.8 MB/s | 3.1 MB 00:01 2024-03-21T12:42:35.784 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:42:35.785 INFO:teuthology.orchestra.run.smithi043.stdout:Total 9.7 MB/s | 59 MB 00:06 2024-03-21T12:42:35.817 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-03-21T12:42:35.817 INFO:teuthology.orchestra.run.smithi155.stdout:(6/6): ceph-test-19.0.0-2267.g960fbbe4.el9.x86_ 15 MB/s | 48 MB 00:03 2024-03-21T12:42:35.820 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:42:35.820 INFO:teuthology.orchestra.run.smithi155.stdout:Total 15 MB/s | 49 MB 00:03 2024-03-21T12:42:35.835 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-03-21T12:42:35.855 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-03-21T12:42:35.855 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-03-21T12:42:35.857 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-03-21T12:42:35.857 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-03-21T12:42:36.240 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-03-21T12:42:36.241 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-03-21T12:42:36.355 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-03-21T12:42:36.356 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-03-21T12:42:36.818 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-03-21T12:42:36.885 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-21T12:42:36.936 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-21T12:42:36.976 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-21T12:42:37.018 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-21T12:42:37.045 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephsqlite-2:19.0.0-2267.g960fbbe4.el9.x86_64 5/6 2024-03-21T12:42:37.817 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-03-21T12:42:37.900 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-03-21T12:42:37.936 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-03-21T12:42:37.969 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-03-21T12:42:37.992 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-03-21T12:42:38.009 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 4/34 2024-03-21T12:42:38.209 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 5/34 2024-03-21T12:42:38.265 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 5/34 2024-03-21T12:42:38.610 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-03-21T12:42:38.655 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librados2-2:19.0.0-2267.g960fbbe4.el9.x86_64 7/34 2024-03-21T12:42:38.747 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:19.0.0-2267.g960fbbe4.el9.x86_64 7/34 2024-03-21T12:42:38.878 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rados-2:19.0.0-2267.g960fbbe4.el9.x86_64 8/34 2024-03-21T12:42:38.898 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs2-2:19.0.0-2267.g960fbbe4.el9.x86_64 9/34 2024-03-21T12:42:38.960 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs2-2:19.0.0-2267.g960fbbe4.el9.x86_64 9/34 2024-03-21T12:42:38.989 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-03-21T12:42:39.025 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-argparse-2:19.0.0-2267.g960fbbe4.el9. 11/34 2024-03-21T12:42:39.095 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cephfs-2:19.0.0-2267.g960fbbe4.el9.x86_64 12/34 2024-03-21T12:42:39.117 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libradosstriper1-2:19.0.0-2267.g960fbbe4.el9.x86_6 13/34 2024-03-21T12:42:39.171 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2267.g960fbbe4.el9.x86_6 13/34 2024-03-21T12:42:39.199 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 14/34 2024-03-21T12:42:39.309 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-03-21T12:42:39.350 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-03-21T12:42:39.390 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-03-21T12:42:39.423 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-03-21T12:42:39.452 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-03-21T12:42:39.485 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-03-21T12:42:39.814 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-03-21T12:42:39.839 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librbd1-2:19.0.0-2267.g960fbbe4.el9.x86_64 22/34 2024-03-21T12:42:39.930 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:19.0.0-2267.g960fbbe4.el9.x86_64 22/34 2024-03-21T12:42:39.960 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 23/34 2024-03-21T12:42:39.999 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-03-21T12:42:40.482 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-03-21T12:42:40.517 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librgw2-2:19.0.0-2267.g960fbbe4.el9.x86_64 26/34 2024-03-21T12:42:40.588 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librgw2-2:19.0.0-2267.g960fbbe4.el9.x86_64 26/34 2024-03-21T12:42:40.659 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 27/34 2024-03-21T12:42:40.702 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-common-2:19.0.0-2267.g960fbbe4.el9.x8 28/34 2024-03-21T12:42:41.555 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2267.g960fbbe4.el9.x86_64 5/6 2024-03-21T12:42:41.638 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-test-2:19.0.0-2267.g960fbbe4.el9.x86_64 6/6 2024-03-21T12:42:43.039 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.0.0-2267.g960fbbe4.el9.x86_64 29/34 2024-03-21T12:42:43.082 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-common-2:19.0.0-2267.g960fbbe4.el9.x86_64 29/34 2024-03-21T12:42:43.682 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.0.0-2267.g960fbbe4.el9.x86_64 29/34 2024-03-21T12:42:43.719 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-base-2:19.0.0-2267.g960fbbe4.el9.x86_64 30/34 2024-03-21T12:42:43.808 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-base-2:19.0.0-2267.g960fbbe4.el9.x86_64 30/34 2024-03-21T12:42:43.808 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-21T12:42:43.808 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-21T12:42:43.809 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:42:43.837 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-selinux-2:19.0.0-2267.g960fbbe4.el9.x86_64 31/34 2024-03-21T12:42:43.951 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-test-2:19.0.0-2267.g960fbbe4.el9.x86_64 6/6 2024-03-21T12:42:43.951 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-test-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/6 2024-03-21T12:42:43.951 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephsqlite-2:19.0.0-2267.g960fbbe4.el9.x86_64 2/6 2024-03-21T12:42:43.951 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-21T12:42:43.951 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-21T12:42:43.951 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-21T12:42:44.443 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-21T12:42:44.443 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:42:44.443 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-03-21T12:42:44.443 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-test-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:44.443 INFO:teuthology.orchestra.run.smithi155.stdout: jq-1.6-16.el9.x86_64 2024-03-21T12:42:44.443 INFO:teuthology.orchestra.run.smithi155.stdout: libcephsqlite-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:44.443 INFO:teuthology.orchestra.run.smithi155.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-21T12:42:44.443 INFO:teuthology.orchestra.run.smithi155.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-21T12:42:44.443 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-21T12:42:44.443 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:42:44.443 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:42:44.708 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph 2024-03-21T12:42:45.301 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:00:41 ago on Thu 21 Mar 2024 12:42:04 PM UTC. 2024-03-21T12:42:45.475 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:42:45.478 INFO:teuthology.orchestra.run.smithi155.stdout:========================================================================================= 2024-03-21T12:42:45.478 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-03-21T12:42:45.478 INFO:teuthology.orchestra.run.smithi155.stdout:========================================================================================= 2024-03-21T12:42:45.478 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-03-21T12:42:45.478 INFO:teuthology.orchestra.run.smithi155.stdout: ceph x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 6.6 k 2024-03-21T12:42:45.478 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-03-21T12:42:45.478 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 2.3 M 2024-03-21T12:42:45.478 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 1.5 M 2024-03-21T12:42:45.478 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 248 k 2024-03-21T12:42:45.478 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 4.7 M 2024-03-21T12:42:45.479 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 17 M 2024-03-21T12:42:45.479 INFO:teuthology.orchestra.run.smithi155.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-21T12:42:45.479 INFO:teuthology.orchestra.run.smithi155.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-21T12:42:45.479 INFO:teuthology.orchestra.run.smithi155.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-21T12:42:45.479 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-21T12:42:45.479 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-21T12:42:45.479 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-21T12:42:45.479 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-21T12:42:45.479 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-21T12:42:45.479 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-21T12:42:45.479 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-21T12:42:45.479 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-21T12:42:45.479 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-21T12:42:45.479 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-21T12:42:45.479 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-21T12:42:45.479 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-21T12:42:45.480 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-21T12:42:45.480 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-21T12:42:45.480 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-21T12:42:45.480 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-21T12:42:45.480 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-21T12:42:45.480 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-21T12:42:45.480 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-21T12:42:45.480 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-21T12:42:45.480 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-21T12:42:45.480 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-21T12:42:45.480 INFO:teuthology.orchestra.run.smithi155.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-21T12:42:45.480 INFO:teuthology.orchestra.run.smithi155.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-21T12:42:45.480 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-21T12:42:45.480 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-21T12:42:45.480 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-21T12:42:45.480 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:42:45.481 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-03-21T12:42:45.481 INFO:teuthology.orchestra.run.smithi155.stdout:========================================================================================= 2024-03-21T12:42:45.481 INFO:teuthology.orchestra.run.smithi155.stdout:Install 36 Packages 2024-03-21T12:42:45.481 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:42:45.481 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 30 M 2024-03-21T12:42:45.481 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 107 M 2024-03-21T12:42:45.481 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-03-21T12:42:46.672 INFO:teuthology.orchestra.run.smithi155.stdout:(1/36): ceph-19.0.0-2267.g960fbbe4.el9.x86_64.r 22 kB/s | 6.6 kB 00:00 2024-03-21T12:42:46.855 INFO:teuthology.orchestra.run.smithi155.stdout:(2/36): ceph-mds-19.0.0-2267.g960fbbe4.el9.x86_ 4.8 MB/s | 2.3 MB 00:00 2024-03-21T12:42:47.272 INFO:teuthology.orchestra.run.smithi155.stdout:(3/36): ceph-mgr-19.0.0-2267.g960fbbe4.el9.x86_ 1.7 MB/s | 1.5 MB 00:00 2024-03-21T12:42:47.414 INFO:teuthology.orchestra.run.smithi155.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2267.g960f 1.7 MB/s | 248 kB 00:00 2024-03-21T12:42:47.590 INFO:teuthology.orchestra.run.smithi155.stdout:(5/36): lua-5.4.4-4.el9.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2024-03-21T12:42:47.640 INFO:teuthology.orchestra.run.smithi155.stdout:(6/36): python3-mako-1.1.4-6.el9.noarch.rpm 2.8 MB/s | 144 kB 00:00 2024-03-21T12:42:47.682 INFO:teuthology.orchestra.run.smithi155.stdout:(7/36): python3-markupsafe-1.1.1-12.el9.x86_64. 756 kB/s | 32 kB 00:00 2024-03-21T12:42:47.755 INFO:teuthology.orchestra.run.smithi155.stdout:(8/36): python3-pytz-2021.1-5.el9.noarch.rpm 654 kB/s | 47 kB 00:00 2024-03-21T12:42:47.791 INFO:teuthology.orchestra.run.smithi155.stdout:(9/36): python3-toml-0.10.2-6.el9.noarch.rpm 1.1 MB/s | 39 kB 00:00 2024-03-21T12:42:47.850 INFO:teuthology.orchestra.run.smithi155.stdout:(10/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.0 MB/s | 241 kB 00:00 2024-03-21T12:42:48.017 INFO:teuthology.orchestra.run.smithi155.stdout:(11/36): ceph-osd-19.0.0-2267.g960fbbe4.el9.x86 14 MB/s | 17 MB 00:01 2024-03-21T12:42:48.125 INFO:teuthology.orchestra.run.smithi155.stdout:(12/36): python3-cryptography-36.0.1-4.el9.x86_ 4.2 MB/s | 1.1 MB 00:00 2024-03-21T12:42:48.150 INFO:teuthology.orchestra.run.smithi155.stdout:(13/36): python3-ply-3.11-14.el9.noarch.rpm 777 kB/s | 103 kB 00:00 2024-03-21T12:42:48.217 INFO:teuthology.orchestra.run.smithi155.stdout:(14/36): ceph-mon-19.0.0-2267.g960fbbe4.el9.x86 3.0 MB/s | 4.7 MB 00:01 2024-03-21T12:42:48.245 INFO:teuthology.orchestra.run.smithi155.stdout:(15/36): python3-pycparser-2.20-6.el9.noarch.rp 1.0 MB/s | 124 kB 00:00 2024-03-21T12:42:48.268 INFO:teuthology.orchestra.run.smithi155.stdout:(16/36): python3-urllib3-1.26.5-5.el9.noarch.rp 1.6 MB/s | 187 kB 00:00 2024-03-21T12:42:48.352 INFO:teuthology.orchestra.run.smithi155.stdout:(17/36): lua-devel-5.4.4-4.el9.x86_64.rpm 268 kB/s | 22 kB 00:00 2024-03-21T12:42:48.377 INFO:teuthology.orchestra.run.smithi155.stdout:(18/36): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-03-21T12:42:48.427 INFO:teuthology.orchestra.run.smithi155.stdout:(19/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-03-21T12:42:48.453 INFO:teuthology.orchestra.run.smithi155.stdout:(20/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 433 kB/s | 43 kB 00:00 2024-03-21T12:42:48.478 INFO:teuthology.orchestra.run.smithi155.stdout:(21/36): python3-cherrypy-18.6.1-2.el9.noarch.r 7.0 MB/s | 358 kB 00:00 2024-03-21T12:42:48.503 INFO:teuthology.orchestra.run.smithi155.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 213 kB/s | 11 kB 00:00 2024-03-21T12:42:48.528 INFO:teuthology.orchestra.run.smithi155.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 353 kB/s | 18 kB 00:00 2024-03-21T12:42:48.545 INFO:teuthology.orchestra.run.smithi155.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 554 kB/s | 23 kB 00:00 2024-03-21T12:42:48.562 INFO:teuthology.orchestra.run.smithi155.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 579 kB/s | 19 kB 00:00 2024-03-21T12:42:48.579 INFO:teuthology.orchestra.run.smithi155.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 587 kB/s | 20 kB 00:00 2024-03-21T12:42:48.596 INFO:teuthology.orchestra.run.smithi155.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 1.4 MB/s | 46 kB 00:00 2024-03-21T12:42:48.630 INFO:teuthology.orchestra.run.smithi155.stdout:(28/36): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-03-21T12:42:48.655 INFO:teuthology.orchestra.run.smithi155.stdout:(29/36): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-03-21T12:42:48.680 INFO:teuthology.orchestra.run.smithi155.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 326 kB/s | 16 kB 00:00 2024-03-21T12:42:48.705 INFO:teuthology.orchestra.run.smithi155.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-03-21T12:42:48.731 INFO:teuthology.orchestra.run.smithi155.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 715 kB/s | 36 kB 00:00 2024-03-21T12:42:48.756 INFO:teuthology.orchestra.run.smithi155.stdout:(33/36): python3-requests-2.25.1-8.el9.noarch.r 232 kB/s | 125 kB 00:00 2024-03-21T12:42:48.782 INFO:teuthology.orchestra.run.smithi155.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-03-21T12:42:48.807 INFO:teuthology.orchestra.run.smithi155.stdout:(35/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 5.5 MB/s | 427 kB 00:00 2024-03-21T12:42:48.832 INFO:teuthology.orchestra.run.smithi155.stdout:(36/36): python3-zc-lockfile-2.0-10.el9.noarch. 266 kB/s | 20 kB 00:00 2024-03-21T12:42:48.834 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:42:48.834 INFO:teuthology.orchestra.run.smithi155.stdout:Total 8.9 MB/s | 30 MB 00:03 2024-03-21T12:42:48.859 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-03-21T12:42:48.902 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-03-21T12:42:48.902 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-03-21T12:42:49.301 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-03-21T12:42:49.301 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-03-21T12:42:50.618 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-03-21T12:42:50.685 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-03-21T12:42:50.727 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-03-21T12:42:50.754 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-03-21T12:42:50.824 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-03-21T12:42:50.861 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-03-21T12:42:50.893 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-03-21T12:42:50.986 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-03-21T12:42:51.090 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-03-21T12:42:51.147 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-03-21T12:42:51.219 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-03-21T12:42:51.273 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-03-21T12:42:51.311 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-03-21T12:42:51.360 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-03-21T12:42:51.416 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-03-21T12:42:51.602 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-03-21T12:42:51.664 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-03-21T12:42:51.709 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-03-21T12:42:51.747 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-03-21T12:42:51.777 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-03-21T12:42:51.854 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-03-21T12:42:51.888 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-03-21T12:42:51.921 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-03-21T12:42:51.961 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-03-21T12:42:51.997 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-03-21T12:42:52.031 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-03-21T12:42:52.121 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-03-21T12:42:52.162 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-03-21T12:42:52.235 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-03-21T12:42:52.393 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-03-21T12:42:52.514 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-03-21T12:42:52.672 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2267.g960fbbe4.el9. 31/36 2024-03-21T12:42:52.709 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-2:19.0.0-2267.g960fbbe4.el9.x86_64 32/36 2024-03-21T12:42:52.735 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2267.g960fbbe4.el9.x86_64 32/36 2024-03-21T12:42:52.735 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T12:42:52.735 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-21T12:42:52.735 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T12:42:52.735 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T12:42:52.735 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:42:53.910 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2267.g960fbbe4.el9.x86_64 31/34 2024-03-21T12:42:53.911 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /sys 2024-03-21T12:42:53.911 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /proc 2024-03-21T12:42:53.911 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /mnt 2024-03-21T12:42:53.911 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /var/tmp 2024-03-21T12:42:53.911 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /home 2024-03-21T12:42:53.911 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /root 2024-03-21T12:42:53.911 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /tmp 2024-03-21T12:42:53.911 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:42:54.168 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-osd-2:19.0.0-2267.g960fbbe4.el9.x86_64 33/36 2024-03-21T12:42:54.195 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-osd-2:19.0.0-2267.g960fbbe4.el9.x86_64 33/36 2024-03-21T12:42:54.195 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T12:42:54.195 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-21T12:42:54.195 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T12:42:54.195 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T12:42:54.195 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:42:54.636 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mon-2:19.0.0-2267.g960fbbe4.el9.x86_64 34/36 2024-03-21T12:42:54.659 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mon-2:19.0.0-2267.g960fbbe4.el9.x86_64 34/36 2024-03-21T12:42:54.659 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T12:42:54.659 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-21T12:42:54.659 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T12:42:54.659 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T12:42:54.659 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:42:54.865 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-radosgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 32/34 2024-03-21T12:42:54.886 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mds-2:19.0.0-2267.g960fbbe4.el9.x86_64 35/36 2024-03-21T12:42:54.888 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 32/34 2024-03-21T12:42:54.889 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T12:42:54.889 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-21T12:42:54.889 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T12:42:54.889 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T12:42:54.889 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:42:54.891 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-21T12:42:54.910 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mds-2:19.0.0-2267.g960fbbe4.el9.x86_64 35/36 2024-03-21T12:42:54.910 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T12:42:54.910 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-21T12:42:54.910 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T12:42:54.910 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T12:42:54.910 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:42:54.967 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-03-21T12:42:54.967 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-21T12:42:54.982 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-2:19.0.0-2267.g960fbbe4.el9.x86_64 36/36 2024-03-21T12:42:56.091 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-03-21T12:42:56.091 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-base-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/34 2024-03-21T12:42:56.091 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-common-2:19.0.0-2267.g960fbbe4.el9.x86_64 2/34 2024-03-21T12:42:56.091 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-radosgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 3/34 2024-03-21T12:42:56.091 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-selinux-2:19.0.0-2267.g960fbbe4.el9.x86_64 4/34 2024-03-21T12:42:56.091 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs2-2:19.0.0-2267.g960fbbe4.el9.x86_64 5/34 2024-03-21T12:42:56.091 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libradosstriper1-2:19.0.0-2267.g960fbbe4.el9.x86_6 6/34 2024-03-21T12:42:56.091 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librgw2-2:19.0.0-2267.g960fbbe4.el9.x86_64 7/34 2024-03-21T12:42:56.091 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2267.g960fbbe4.el9. 8/34 2024-03-21T12:42:56.091 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-common-2:19.0.0-2267.g960fbbe4.el9.x8 9/34 2024-03-21T12:42:56.092 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cephfs-2:19.0.0-2267.g960fbbe4.el9.x86_64 10/34 2024-03-21T12:42:56.092 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rados-2:19.0.0-2267.g960fbbe4.el9.x86_64 11/34 2024-03-21T12:42:56.092 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 12/34 2024-03-21T12:42:56.092 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 13/34 2024-03-21T12:42:56.092 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-03-21T12:42:56.092 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-03-21T12:42:56.092 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-03-21T12:42:56.092 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-03-21T12:42:56.092 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-03-21T12:42:56.092 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-03-21T12:42:56.092 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 20/34 2024-03-21T12:42:56.092 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 21/34 2024-03-21T12:42:56.092 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 22/34 2024-03-21T12:42:56.092 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-03-21T12:42:56.093 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-03-21T12:42:56.093 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-03-21T12:42:56.093 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-03-21T12:42:56.093 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-03-21T12:42:56.093 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-03-21T12:42:56.093 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-03-21T12:42:56.093 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-03-21T12:42:56.093 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:19.0.0-2267.g960fbbe4.el9.x86_64 31/34 2024-03-21T12:42:56.093 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-03-21T12:42:56.093 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:19.0.0-2267.g960fbbe4.el9.x86_64 33/34 2024-03-21T12:42:56.807 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-2:19.0.0-2267.g960fbbe4.el9.x86_64 36/36 2024-03-21T12:42:56.807 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/36 2024-03-21T12:42:56.807 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mds-2:19.0.0-2267.g960fbbe4.el9.x86_64 2/36 2024-03-21T12:42:56.807 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-2:19.0.0-2267.g960fbbe4.el9.x86_64 3/36 2024-03-21T12:42:56.807 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mon-2:19.0.0-2267.g960fbbe4.el9.x86_64 4/36 2024-03-21T12:42:56.807 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-osd-2:19.0.0-2267.g960fbbe4.el9.x86_64 5/36 2024-03-21T12:42:56.807 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2267.g960fbbe4.el9. 6/36 2024-03-21T12:42:56.807 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-03-21T12:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-03-21T12:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-03-21T12:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-03-21T12:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-03-21T12:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-03-21T12:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-03-21T12:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-03-21T12:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-03-21T12:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/36 2024-03-21T12:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 17/36 2024-03-21T12:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-03-21T12:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-03-21T12:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-03-21T12:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-03-21T12:42:56.808 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-03-21T12:42:56.809 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-03-21T12:42:56.809 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-03-21T12:42:56.809 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-03-21T12:42:56.810 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-03-21T12:42:56.810 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-03-21T12:42:56.810 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-03-21T12:42:56.810 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-03-21T12:42:56.810 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-03-21T12:42:56.810 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-03-21T12:42:56.810 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-03-21T12:42:56.810 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-03-21T12:42:56.811 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-03-21T12:42:56.811 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-03-21T12:42:57.116 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-03-21T12:42:57.116 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:42:57.116 INFO:teuthology.orchestra.run.smithi043.stdout:Upgraded: 2024-03-21T12:42:57.116 INFO:teuthology.orchestra.run.smithi043.stdout: librados2-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:57.116 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:57.116 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-03-21T12:42:57.116 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-21T12:42:57.116 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:57.116 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:57.117 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:57.117 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:57.117 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-21T12:42:57.117 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-21T12:42:57.117 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-21T12:42:57.117 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:57.117 INFO:teuthology.orchestra.run.smithi043.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-21T12:42:57.117 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-21T12:42:57.117 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-21T12:42:57.117 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:57.117 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-21T12:42:57.117 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:57.117 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-21T12:42:57.117 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-21T12:42:57.117 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-21T12:42:57.118 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:57.118 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:57.118 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:57.118 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-21T12:42:57.118 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:57.118 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:57.118 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:57.118 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-21T12:42:57.118 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-21T12:42:57.118 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-21T12:42:57.118 INFO:teuthology.orchestra.run.smithi043.stdout: re2-1:20211101-3.el9.x86_64 2024-03-21T12:42:57.118 INFO:teuthology.orchestra.run.smithi043.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-21T12:42:57.118 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:42:57.119 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:42:57.458 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-test 2024-03-21T12:42:57.784 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-03-21T12:42:57.784 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:42:57.784 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-03-21T12:42:57.784 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:57.784 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mds-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:57.785 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:57.785 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T12:42:57.785 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mon-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:57.785 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-osd-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:42:57.785 INFO:teuthology.orchestra.run.smithi155.stdout: lua-5.4.4-4.el9.x86_64 2024-03-21T12:42:57.785 INFO:teuthology.orchestra.run.smithi155.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-21T12:42:57.785 INFO:teuthology.orchestra.run.smithi155.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-21T12:42:57.785 INFO:teuthology.orchestra.run.smithi155.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-21T12:42:57.785 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-21T12:42:57.785 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-21T12:42:57.785 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-21T12:42:57.785 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-21T12:42:57.785 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-21T12:42:57.785 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-21T12:42:57.785 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-21T12:42:57.785 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-21T12:42:57.786 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-21T12:42:57.786 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-21T12:42:57.786 INFO:teuthology.orchestra.run.smithi155.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-21T12:42:57.786 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-21T12:42:57.786 INFO:teuthology.orchestra.run.smithi155.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-21T12:42:57.786 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-21T12:42:57.786 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ply-3.11-14.el9.noarch 2024-03-21T12:42:57.786 INFO:teuthology.orchestra.run.smithi155.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-21T12:42:57.786 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-21T12:42:57.786 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-21T12:42:57.786 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-21T12:42:57.786 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-21T12:42:57.786 INFO:teuthology.orchestra.run.smithi155.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-21T12:42:57.786 INFO:teuthology.orchestra.run.smithi155.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-21T12:42:57.786 INFO:teuthology.orchestra.run.smithi155.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-21T12:42:57.787 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-21T12:42:57.787 INFO:teuthology.orchestra.run.smithi155.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-21T12:42:57.787 INFO:teuthology.orchestra.run.smithi155.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-21T12:42:57.787 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:42:57.787 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:42:58.027 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:31 ago on Thu 21 Mar 2024 12:42:27 PM UTC. 2024-03-21T12:42:58.144 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-base 2024-03-21T12:42:58.185 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:42:58.186 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:42:58.186 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-03-21T12:42:58.186 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:42:58.186 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-03-21T12:42:58.186 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 48 M 2024-03-21T12:42:58.186 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-03-21T12:42:58.187 INFO:teuthology.orchestra.run.smithi043.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-21T12:42:58.187 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 163 k 2024-03-21T12:42:58.187 INFO:teuthology.orchestra.run.smithi043.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-21T12:42:58.187 INFO:teuthology.orchestra.run.smithi043.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-21T12:42:58.187 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-21T12:42:58.187 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:42:58.187 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-03-21T12:42:58.187 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:42:58.187 INFO:teuthology.orchestra.run.smithi043.stdout:Install 6 Packages 2024-03-21T12:42:58.187 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:42:58.188 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 49 M 2024-03-21T12:42:58.188 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 206 M 2024-03-21T12:42:58.188 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-03-21T12:42:58.532 INFO:teuthology.orchestra.run.smithi043.stdout:(1/6): oniguruma-6.9.6-1.el9.5.x86_64.rpm 1.1 MB/s | 217 kB 00:00 2024-03-21T12:42:58.640 INFO:teuthology.orchestra.run.smithi043.stdout:(2/6): socat-1.7.4.1-5.el9.x86_64.rpm 2.7 MB/s | 300 kB 00:00 2024-03-21T12:42:58.682 INFO:teuthology.orchestra.run.smithi043.stdout:(3/6): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 63 kB 00:00 2024-03-21T12:42:58.707 INFO:teuthology.orchestra.run.smithi043.stdout:(4/6): libcephsqlite-19.0.0-2267.g960fbbe4.el9. 435 kB/s | 163 kB 00:00 2024-03-21T12:42:58.718 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:00:54 ago on Thu 21 Mar 2024 12:42:04 PM UTC. 2024-03-21T12:42:58.815 INFO:teuthology.orchestra.run.smithi155.stdout:Package ceph-base-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T12:42:58.867 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:42:58.868 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-03-21T12:42:58.868 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:42:58.899 INFO:teuthology.orchestra.run.smithi043.stdout:(5/6): jq-1.6-16.el9.x86_64.rpm 864 kB/s | 187 kB 00:00 2024-03-21T12:42:58.949 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install cephadm 2024-03-21T12:42:59.513 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:00:55 ago on Thu 21 Mar 2024 12:42:04 PM UTC. 2024-03-21T12:42:59.663 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:42:59.663 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:42:59.663 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-03-21T12:42:59.663 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:42:59.663 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-03-21T12:42:59.664 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 750 k 2024-03-21T12:42:59.664 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:42:59.664 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-03-21T12:42:59.664 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:42:59.664 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-03-21T12:42:59.664 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:42:59.664 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 750 k 2024-03-21T12:42:59.664 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 757 k 2024-03-21T12:42:59.664 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-03-21T12:43:00.616 INFO:teuthology.orchestra.run.smithi155.stdout:cephadm-19.0.0-2267.g960fbbe4.el9.noarch.rpm 789 kB/s | 750 kB 00:00 2024-03-21T12:43:00.616 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:43:00.617 INFO:teuthology.orchestra.run.smithi155.stdout:Total 787 kB/s | 750 kB 00:00 2024-03-21T12:43:00.617 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-03-21T12:43:00.623 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-03-21T12:43:00.623 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-03-21T12:43:00.629 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-03-21T12:43:00.629 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-03-21T12:43:00.704 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-03-21T12:43:01.066 INFO:teuthology.orchestra.run.smithi043.stdout:(6/6): ceph-test-19.0.0-2267.g960fbbe4.el9.x86_ 17 MB/s | 48 MB 00:02 2024-03-21T12:43:01.069 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:43:01.069 INFO:teuthology.orchestra.run.smithi043.stdout:Total 17 MB/s | 49 MB 00:02 2024-03-21T12:43:01.084 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-03-21T12:43:01.107 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-03-21T12:43:01.107 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-03-21T12:43:01.174 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 1/1 2024-03-21T12:43:01.254 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 1/1 2024-03-21T12:43:01.506 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-03-21T12:43:01.506 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-03-21T12:43:01.623 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 1/1 2024-03-21T12:43:01.912 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 1/1 2024-03-21T12:43:01.912 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:01.912 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-03-21T12:43:01.912 INFO:teuthology.orchestra.run.smithi155.stdout: cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T12:43:01.912 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:01.912 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:43:02.105 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-03-21T12:43:02.121 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-immutable-object-cache 2024-03-21T12:43:02.177 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/6 2024-03-21T12:43:02.224 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/6 2024-03-21T12:43:02.263 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-21T12:43:02.309 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : jq-1.6-16.el9.x86_64 4/6 2024-03-21T12:43:02.344 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephsqlite-2:19.0.0-2267.g960fbbe4.el9.x86_64 5/6 2024-03-21T12:43:02.692 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:00:58 ago on Thu 21 Mar 2024 12:42:04 PM UTC. 2024-03-21T12:43:02.845 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:43:02.846 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:43:02.846 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repo Size 2024-03-21T12:43:02.846 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:43:02.846 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-03-21T12:43:02.846 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 145 k 2024-03-21T12:43:02.846 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:02.847 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-03-21T12:43:02.847 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:43:02.847 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-03-21T12:43:02.847 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:02.847 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 145 k 2024-03-21T12:43:02.847 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 436 k 2024-03-21T12:43:02.847 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-03-21T12:43:03.373 INFO:teuthology.orchestra.run.smithi155.stdout:ceph-immutable-object-cache-19.0.0-2267.g960fbb 276 kB/s | 145 kB 00:00 2024-03-21T12:43:03.374 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:43:03.374 INFO:teuthology.orchestra.run.smithi155.stdout:Total 275 kB/s | 145 kB 00:00 2024-03-21T12:43:03.374 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-03-21T12:43:03.381 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-03-21T12:43:03.382 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-03-21T12:43:03.433 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-03-21T12:43:03.433 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-03-21T12:43:03.592 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-03-21T12:43:03.649 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2267.g960fbbe4. 1/1 2024-03-21T12:43:03.673 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2267.g960fbbe4. 1/1 2024-03-21T12:43:03.673 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T12:43:03.674 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-21T12:43:03.674 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:04.685 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2267.g960fbbe4. 1/1 2024-03-21T12:43:04.685 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:04.685 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-03-21T12:43:04.685 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-immutable-object-cache-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:43:04.685 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:04.685 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:43:04.956 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-mgr 2024-03-21T12:43:05.522 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:01 ago on Thu 21 Mar 2024 12:42:04 PM UTC. 2024-03-21T12:43:05.622 INFO:teuthology.orchestra.run.smithi155.stdout:Package ceph-mgr-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T12:43:05.674 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:43:05.675 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-03-21T12:43:05.676 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:43:05.753 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-mgr-dashboard 2024-03-21T12:43:06.315 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:02 ago on Thu 21 Mar 2024 12:42:04 PM UTC. 2024-03-21T12:43:06.464 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:43:06.465 INFO:teuthology.orchestra.run.smithi155.stdout:====================================================================================== 2024-03-21T12:43:06.465 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-03-21T12:43:06.465 INFO:teuthology.orchestra.run.smithi155.stdout:====================================================================================== 2024-03-21T12:43:06.465 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-03-21T12:43:06.465 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 3.5 M 2024-03-21T12:43:06.465 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-03-21T12:43:06.465 INFO:teuthology.orchestra.run.smithi155.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-21T12:43:06.465 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 34 k 2024-03-21T12:43:06.465 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 16 k 2024-03-21T12:43:06.466 INFO:teuthology.orchestra.run.smithi155.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-21T12:43:06.466 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-21T12:43:06.466 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-21T12:43:06.466 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-21T12:43:06.466 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-21T12:43:06.466 INFO:teuthology.orchestra.run.smithi155.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-21T12:43:06.466 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-21T12:43:06.466 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-21T12:43:06.466 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:06.466 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-03-21T12:43:06.466 INFO:teuthology.orchestra.run.smithi155.stdout:====================================================================================== 2024-03-21T12:43:06.466 INFO:teuthology.orchestra.run.smithi155.stdout:Install 12 Packages 2024-03-21T12:43:06.467 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:06.467 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 8.6 M 2024-03-21T12:43:06.467 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 103 M 2024-03-21T12:43:06.467 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-03-21T12:43:07.002 INFO:teuthology.orchestra.run.smithi155.stdout:(1/12): ceph-grafana-dashboards-19.0.0-2267.g96 142 kB/s | 34 kB 00:00 2024-03-21T12:43:07.044 INFO:teuthology.orchestra.run.smithi155.stdout:(2/12): ceph-prometheus-alerts-19.0.0-2267.g960 59 kB/s | 16 kB 00:00 2024-03-21T12:43:07.157 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2267.g960fbbe4.el9.x86_64 5/6 2024-03-21T12:43:07.194 INFO:teuthology.orchestra.run.smithi155.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.5 MB/s | 238 kB 00:00 2024-03-21T12:43:07.236 INFO:teuthology.orchestra.run.smithi155.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.3 MB/s | 1.0 MB 00:00 2024-03-21T12:43:07.237 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-test-2:19.0.0-2267.g960fbbe4.el9.x86_64 6/6 2024-03-21T12:43:07.303 INFO:teuthology.orchestra.run.smithi155.stdout:(5/12): ceph-mgr-dashboard-19.0.0-2267.g960fbbe 6.5 MB/s | 3.5 MB 00:00 2024-03-21T12:43:07.553 INFO:teuthology.orchestra.run.smithi155.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 86 kB/s | 19 kB 00:00 2024-03-21T12:43:07.678 INFO:teuthology.orchestra.run.smithi155.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.2 MB/s | 548 kB 00:00 2024-03-21T12:43:07.787 INFO:teuthology.orchestra.run.smithi155.stdout:(8/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 8.7 MB/s | 2.0 MB 00:00 2024-03-21T12:43:07.812 INFO:teuthology.orchestra.run.smithi155.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 1.1 MB/s | 144 kB 00:00 2024-03-21T12:43:07.854 INFO:teuthology.orchestra.run.smithi155.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 461 kB/s | 31 kB 00:00 2024-03-21T12:43:07.880 INFO:teuthology.orchestra.run.smithi155.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.7 MB/s | 188 kB 00:00 2024-03-21T12:43:08.963 INFO:teuthology.orchestra.run.smithi155.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 488 kB/s | 863 kB 00:01 2024-03-21T12:43:08.971 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:43:08.971 INFO:teuthology.orchestra.run.smithi155.stdout:Total 3.4 MB/s | 8.6 MB 00:02 2024-03-21T12:43:08.991 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-03-21T12:43:09.010 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-03-21T12:43:09.010 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-03-21T12:43:09.211 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-03-21T12:43:09.212 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-03-21T12:43:09.689 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-test-2:19.0.0-2267.g960fbbe4.el9.x86_64 6/6 2024-03-21T12:43:09.689 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-test-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/6 2024-03-21T12:43:09.689 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephsqlite-2:19.0.0-2267.g960fbbe4.el9.x86_64 2/6 2024-03-21T12:43:09.689 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 3/6 2024-03-21T12:43:09.689 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 4/6 2024-03-21T12:43:09.689 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 5/6 2024-03-21T12:43:09.729 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-03-21T12:43:09.814 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-03-21T12:43:09.849 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-03-21T12:43:09.895 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-03-21T12:43:09.926 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-03-21T12:43:10.039 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-03-21T12:43:10.102 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : jq-1.6-16.el9.x86_64 6/6 2024-03-21T12:43:10.103 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:10.103 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-03-21T12:43:10.103 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:43:10.103 INFO:teuthology.orchestra.run.smithi043.stdout: jq-1.6-16.el9.x86_64 2024-03-21T12:43:10.103 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:43:10.103 INFO:teuthology.orchestra.run.smithi043.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-21T12:43:10.103 INFO:teuthology.orchestra.run.smithi043.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-21T12:43:10.103 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-21T12:43:10.103 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:10.103 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:43:10.161 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-03-21T12:43:10.247 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-03-21T12:43:10.298 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-03-21T12:43:10.326 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-03-21T12:43:10.329 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph 2024-03-21T12:43:10.358 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2267.g960fbbe4.el9 10/12 2024-03-21T12:43:10.902 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:43 ago on Thu 21 Mar 2024 12:42:27 PM UTC. 2024-03-21T12:43:11.059 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:43:11.061 INFO:teuthology.orchestra.run.smithi043.stdout:========================================================================================= 2024-03-21T12:43:11.061 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-03-21T12:43:11.062 INFO:teuthology.orchestra.run.smithi043.stdout:========================================================================================= 2024-03-21T12:43:11.062 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-03-21T12:43:11.062 INFO:teuthology.orchestra.run.smithi043.stdout: ceph x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 6.6 k 2024-03-21T12:43:11.062 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-03-21T12:43:11.062 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 2.3 M 2024-03-21T12:43:11.062 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 1.5 M 2024-03-21T12:43:11.062 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 248 k 2024-03-21T12:43:11.062 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 4.7 M 2024-03-21T12:43:11.062 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 17 M 2024-03-21T12:43:11.062 INFO:teuthology.orchestra.run.smithi043.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-21T12:43:11.062 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-21T12:43:11.062 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-21T12:43:11.063 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-21T12:43:11.063 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-21T12:43:11.063 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-21T12:43:11.063 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-21T12:43:11.063 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-21T12:43:11.063 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-21T12:43:11.063 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-21T12:43:11.063 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-21T12:43:11.063 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-21T12:43:11.063 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-21T12:43:11.063 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-21T12:43:11.063 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-21T12:43:11.064 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-21T12:43:11.064 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-21T12:43:11.064 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-21T12:43:11.064 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-21T12:43:11.064 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-21T12:43:11.064 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-21T12:43:11.064 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-21T12:43:11.064 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-21T12:43:11.064 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-21T12:43:11.064 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-21T12:43:11.064 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-21T12:43:11.064 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-21T12:43:11.064 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-21T12:43:11.065 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-21T12:43:11.065 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-21T12:43:11.065 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:11.065 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-03-21T12:43:11.065 INFO:teuthology.orchestra.run.smithi043.stdout:========================================================================================= 2024-03-21T12:43:11.065 INFO:teuthology.orchestra.run.smithi043.stdout:Install 36 Packages 2024-03-21T12:43:11.065 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:11.066 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 30 M 2024-03-21T12:43:11.066 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 107 M 2024-03-21T12:43:11.066 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-03-21T12:43:11.410 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2267.g960fbbe4.el 11/12 2024-03-21T12:43:11.451 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2267.g960fbbe4.el9.noa 12/12 2024-03-21T12:43:11.736 INFO:teuthology.orchestra.run.smithi043.stdout:(1/36): ceph-19.0.0-2267.g960fbbe4.el9.x86_64.r 41 kB/s | 6.6 kB 00:00 2024-03-21T12:43:11.994 INFO:teuthology.orchestra.run.smithi043.stdout:(2/36): ceph-mds-19.0.0-2267.g960fbbe4.el9.x86_ 5.5 MB/s | 2.3 MB 00:00 2024-03-21T12:43:12.044 INFO:teuthology.orchestra.run.smithi043.stdout:(3/36): ceph-mgr-19.0.0-2267.g960fbbe4.el9.x86_ 3.2 MB/s | 1.5 MB 00:00 2024-03-21T12:43:12.145 INFO:teuthology.orchestra.run.smithi043.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2267.g960f 2.4 MB/s | 248 kB 00:00 2024-03-21T12:43:12.353 INFO:teuthology.orchestra.run.smithi043.stdout:(5/36): lua-5.4.4-4.el9.x86_64.rpm 897 kB/s | 187 kB 00:00 2024-03-21T12:43:12.462 INFO:teuthology.orchestra.run.smithi043.stdout:(6/36): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 144 kB 00:00 2024-03-21T12:43:12.520 INFO:teuthology.orchestra.run.smithi043.stdout:(7/36): python3-markupsafe-1.1.1-12.el9.x86_64. 542 kB/s | 32 kB 00:00 2024-03-21T12:43:12.579 INFO:teuthology.orchestra.run.smithi043.stdout:(8/36): python3-pytz-2021.1-5.el9.noarch.rpm 814 kB/s | 47 kB 00:00 2024-03-21T12:43:12.637 INFO:teuthology.orchestra.run.smithi043.stdout:(9/36): python3-toml-0.10.2-6.el9.noarch.rpm 660 kB/s | 39 kB 00:00 2024-03-21T12:43:12.713 INFO:teuthology.orchestra.run.smithi043.stdout:(10/36): ceph-mon-19.0.0-2267.g960fbbe4.el9.x86 4.8 MB/s | 4.7 MB 00:00 2024-03-21T12:43:12.796 INFO:teuthology.orchestra.run.smithi043.stdout:(11/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.5 MB/s | 241 kB 00:00 2024-03-21T12:43:12.838 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2267.g960fbbe4.el9.noa 12/12 2024-03-21T12:43:12.838 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2267.g960fbbe4.el 1/12 2024-03-21T12:43:12.839 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2267.g960fbbe4.el9.noa 2/12 2024-03-21T12:43:12.839 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2267.g960fbbe4.el9 3/12 2024-03-21T12:43:12.839 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-03-21T12:43:12.839 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-03-21T12:43:12.839 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-03-21T12:43:12.839 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-03-21T12:43:12.839 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-03-21T12:43:12.839 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-03-21T12:43:12.839 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-03-21T12:43:12.839 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-03-21T12:43:13.030 INFO:teuthology.orchestra.run.smithi043.stdout:(12/36): ceph-osd-19.0.0-2267.g960fbbe4.el9.x86 16 MB/s | 17 MB 00:01 2024-03-21T12:43:13.197 INFO:teuthology.orchestra.run.smithi043.stdout:(13/36): python3-ply-3.11-14.el9.noarch.rpm 257 kB/s | 103 kB 00:00 2024-03-21T12:43:13.203 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-03-21T12:43:13.203 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:13.203 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-03-21T12:43:13.203 INFO:teuthology.orchestra.run.smithi155.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-21T12:43:13.203 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-grafana-dashboards-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T12:43:13.203 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T12:43:13.203 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-prometheus-alerts-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T12:43:13.203 INFO:teuthology.orchestra.run.smithi155.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-21T12:43:13.204 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-21T12:43:13.204 INFO:teuthology.orchestra.run.smithi155.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-21T12:43:13.204 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-21T12:43:13.204 INFO:teuthology.orchestra.run.smithi155.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-21T12:43:13.204 INFO:teuthology.orchestra.run.smithi155.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-21T12:43:13.204 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-21T12:43:13.204 INFO:teuthology.orchestra.run.smithi155.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-21T12:43:13.204 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:13.204 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:43:13.239 INFO:teuthology.orchestra.run.smithi043.stdout:(14/36): python3-cryptography-36.0.1-4.el9.x86_ 2.2 MB/s | 1.1 MB 00:00 2024-03-21T12:43:13.289 INFO:teuthology.orchestra.run.smithi043.stdout:(15/36): python3-pycparser-2.20-6.el9.noarch.rp 481 kB/s | 124 kB 00:00 2024-03-21T12:43:13.364 INFO:teuthology.orchestra.run.smithi043.stdout:(16/36): python3-urllib3-1.26.5-5.el9.noarch.rp 1.1 MB/s | 187 kB 00:00 2024-03-21T12:43:13.448 INFO:teuthology.orchestra.run.smithi043.stdout:(17/36): lua-devel-5.4.4-4.el9.x86_64.rpm 141 kB/s | 22 kB 00:00 2024-03-21T12:43:13.473 INFO:teuthology.orchestra.run.smithi043.stdout:(18/36): python3-requests-2.25.1-8.el9.noarch.r 534 kB/s | 125 kB 00:00 2024-03-21T12:43:13.502 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-21T12:43:13.790 INFO:teuthology.orchestra.run.smithi043.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 127 kB/s | 43 kB 00:00 2024-03-21T12:43:13.815 INFO:teuthology.orchestra.run.smithi043.stdout:(20/36): luarocks-3.9.2-1.el9.noarch.rpm 335 kB/s | 151 kB 00:00 2024-03-21T12:43:13.899 INFO:teuthology.orchestra.run.smithi043.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 403 kB/s | 172 kB 00:00 2024-03-21T12:43:14.071 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:10 ago on Thu 21 Mar 2024 12:42:04 PM UTC. 2024-03-21T12:43:14.116 INFO:teuthology.orchestra.run.smithi043.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 36 kB/s | 11 kB 00:00 2024-03-21T12:43:14.166 INFO:teuthology.orchestra.run.smithi043.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 66 kB/s | 18 kB 00:00 2024-03-21T12:43:14.222 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:43:14.223 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================================ 2024-03-21T12:43:14.223 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-03-21T12:43:14.223 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================================ 2024-03-21T12:43:14.223 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-03-21T12:43:14.223 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 7.4 M 2024-03-21T12:43:14.223 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-03-21T12:43:14.223 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-21T12:43:14.223 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-21T12:43:14.224 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-21T12:43:14.224 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-21T12:43:14.224 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-21T12:43:14.224 INFO:teuthology.orchestra.run.smithi155.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-21T12:43:14.224 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-21T12:43:14.224 INFO:teuthology.orchestra.run.smithi155.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-21T12:43:14.224 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-21T12:43:14.224 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-21T12:43:14.224 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-21T12:43:14.224 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:14.224 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-03-21T12:43:14.224 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================================ 2024-03-21T12:43:14.224 INFO:teuthology.orchestra.run.smithi155.stdout:Install 12 Packages 2024-03-21T12:43:14.224 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:14.225 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 37 M 2024-03-21T12:43:14.225 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 209 M 2024-03-21T12:43:14.225 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-03-21T12:43:14.383 INFO:teuthology.orchestra.run.smithi043.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 87 kB/s | 23 kB 00:00 2024-03-21T12:43:14.433 INFO:teuthology.orchestra.run.smithi043.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 73 kB/s | 19 kB 00:00 2024-03-21T12:43:14.459 INFO:teuthology.orchestra.run.smithi043.stdout:(26/36): python3-cherrypy-18.6.1-2.el9.noarch.r 536 kB/s | 358 kB 00:00 2024-03-21T12:43:14.651 INFO:teuthology.orchestra.run.smithi043.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 74 kB/s | 20 kB 00:00 2024-03-21T12:43:14.784 INFO:teuthology.orchestra.run.smithi043.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 132 kB/s | 46 kB 00:00 2024-03-21T12:43:14.809 INFO:teuthology.orchestra.run.smithi043.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 225 kB/s | 79 kB 00:00 2024-03-21T12:43:14.905 INFO:teuthology.orchestra.run.smithi155.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 284 kB/s | 32 kB 00:00 2024-03-21T12:43:14.963 INFO:teuthology.orchestra.run.smithi155.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 494 kB/s | 17 kB 00:00 2024-03-21T12:43:14.997 INFO:teuthology.orchestra.run.smithi155.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 34 kB 00:00 2024-03-21T12:43:15.060 INFO:teuthology.orchestra.run.smithi043.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 60 kB/s | 16 kB 00:00 2024-03-21T12:43:15.168 INFO:teuthology.orchestra.run.smithi043.stdout:(31/36): python3-pecan-1.4.2-3.el9.noarch.rpm 526 kB/s | 272 kB 00:00 2024-03-21T12:43:15.181 INFO:teuthology.orchestra.run.smithi155.stdout:(4/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 26 MB/s | 4.7 MB 00:00 2024-03-21T12:43:15.235 INFO:teuthology.orchestra.run.smithi043.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 211 kB/s | 90 kB 00:00 2024-03-21T12:43:15.314 INFO:teuthology.orchestra.run.smithi155.stdout:(5/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 4.8 MB 00:00 2024-03-21T12:43:15.360 INFO:teuthology.orchestra.run.smithi043.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 119 kB/s | 36 kB 00:00 2024-03-21T12:43:15.364 INFO:teuthology.orchestra.run.smithi155.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 5.2 MB/s | 3.0 MB 00:00 2024-03-21T12:43:15.389 INFO:teuthology.orchestra.run.smithi155.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.9 MB/s | 217 kB 00:00 2024-03-21T12:43:15.635 INFO:teuthology.orchestra.run.smithi043.stdout:(34/36): python3-zc-lockfile-2.0-10.el9.noarch. 73 kB/s | 20 kB 00:00 2024-03-21T12:43:15.669 INFO:teuthology.orchestra.run.smithi043.stdout:(35/36): python3-webob-1.8.7-6.el9.noarch.rpm 459 kB/s | 230 kB 00:00 2024-03-21T12:43:15.698 INFO:teuthology.orchestra.run.smithi155.stdout:(8/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 47 MB/s | 16 MB 00:00 2024-03-21T12:43:15.724 INFO:teuthology.orchestra.run.smithi155.stdout:(9/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.4 MB/s | 803 kB 00:00 2024-03-21T12:43:15.749 INFO:teuthology.orchestra.run.smithi155.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-03-21T12:43:15.819 INFO:teuthology.orchestra.run.smithi043.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 731 kB/s | 427 kB 00:00 2024-03-21T12:43:15.824 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:43:15.824 INFO:teuthology.orchestra.run.smithi043.stdout:Total 6.3 MB/s | 30 MB 00:04 2024-03-21T12:43:15.850 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-03-21T12:43:15.894 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-03-21T12:43:15.894 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-03-21T12:43:15.908 INFO:teuthology.orchestra.run.smithi155.stdout:(11/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.3 MB/s | 245 kB 00:00 2024-03-21T12:43:16.314 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-03-21T12:43:16.314 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-03-21T12:43:17.647 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-03-21T12:43:17.705 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-03-21T12:43:17.738 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-03-21T12:43:17.766 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-03-21T12:43:17.839 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-03-21T12:43:17.882 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-03-21T12:43:17.914 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-03-21T12:43:18.009 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-03-21T12:43:18.096 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-03-21T12:43:18.143 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-03-21T12:43:18.217 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-03-21T12:43:18.269 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 11/36 2024-03-21T12:43:18.307 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 12/36 2024-03-21T12:43:18.365 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ply-3.11-14.el9.noarch 13/36 2024-03-21T12:43:18.428 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 14/36 2024-03-21T12:43:18.628 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 15/36 2024-03-21T12:43:18.677 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 16/36 2024-03-21T12:43:18.720 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 17/36 2024-03-21T12:43:18.759 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 18/36 2024-03-21T12:43:18.788 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 19/36 2024-03-21T12:43:18.868 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 20/36 2024-03-21T12:43:18.907 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-03-21T12:43:18.941 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 22/36 2024-03-21T12:43:18.982 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 23/36 2024-03-21T12:43:19.025 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/36 2024-03-21T12:43:19.059 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 25/36 2024-03-21T12:43:19.153 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 26/36 2024-03-21T12:43:19.190 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/36 2024-03-21T12:43:19.266 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 28/36 2024-03-21T12:43:19.416 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 29/36 2024-03-21T12:43:19.548 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-03-21T12:43:19.715 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2267.g960fbbe4.el9. 31/36 2024-03-21T12:43:19.744 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-2:19.0.0-2267.g960fbbe4.el9.x86_64 32/36 2024-03-21T12:43:19.769 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2267.g960fbbe4.el9.x86_64 32/36 2024-03-21T12:43:19.769 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T12:43:19.769 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-21T12:43:19.769 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T12:43:19.769 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T12:43:19.769 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:21.141 INFO:teuthology.orchestra.run.smithi155.stdout:(12/12): ceph-mgr-diskprediction-local-19.0.0-2 1.2 MB/s | 7.4 MB 00:06 2024-03-21T12:43:21.143 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:43:21.143 INFO:teuthology.orchestra.run.smithi155.stdout:Total 5.4 MB/s | 37 MB 00:06 2024-03-21T12:43:21.188 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-03-21T12:43:21.213 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-03-21T12:43:21.213 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-03-21T12:43:21.278 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-osd-2:19.0.0-2267.g960fbbe4.el9.x86_64 33/36 2024-03-21T12:43:21.305 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-osd-2:19.0.0-2267.g960fbbe4.el9.x86_64 33/36 2024-03-21T12:43:21.305 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T12:43:21.305 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-21T12:43:21.305 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T12:43:21.305 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T12:43:21.305 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:21.567 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-03-21T12:43:21.568 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-03-21T12:43:21.762 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mon-2:19.0.0-2267.g960fbbe4.el9.x86_64 34/36 2024-03-21T12:43:21.786 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mon-2:19.0.0-2267.g960fbbe4.el9.x86_64 34/36 2024-03-21T12:43:21.786 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T12:43:21.786 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-21T12:43:21.786 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T12:43:21.786 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T12:43:21.786 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:22.021 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mds-2:19.0.0-2267.g960fbbe4.el9.x86_64 35/36 2024-03-21T12:43:22.043 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mds-2:19.0.0-2267.g960fbbe4.el9.x86_64 35/36 2024-03-21T12:43:22.044 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T12:43:22.044 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-21T12:43:22.044 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T12:43:22.044 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T12:43:22.044 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:22.113 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-2:19.0.0-2267.g960fbbe4.el9.x86_64 36/36 2024-03-21T12:43:22.280 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-03-21T12:43:22.347 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-21T12:43:22.378 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-21T12:43:22.472 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-21T12:43:22.515 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-21T12:43:22.692 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-21T12:43:22.722 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-21T12:43:22.807 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-21T12:43:23.472 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-21T12:43:23.600 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-21T12:43:23.806 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-2:19.0.0-2267.g960fbbe4.el9.x86_64 36/36 2024-03-21T12:43:23.806 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/36 2024-03-21T12:43:23.807 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mds-2:19.0.0-2267.g960fbbe4.el9.x86_64 2/36 2024-03-21T12:43:23.807 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-2:19.0.0-2267.g960fbbe4.el9.x86_64 3/36 2024-03-21T12:43:23.807 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mon-2:19.0.0-2267.g960fbbe4.el9.x86_64 4/36 2024-03-21T12:43:23.807 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-osd-2:19.0.0-2267.g960fbbe4.el9.x86_64 5/36 2024-03-21T12:43:23.807 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2267.g960fbbe4.el9. 6/36 2024-03-21T12:43:23.807 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-5.4.4-4.el9.x86_64 7/36 2024-03-21T12:43:23.807 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 8/36 2024-03-21T12:43:23.807 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 9/36 2024-03-21T12:43:23.807 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 10/36 2024-03-21T12:43:23.807 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 11/36 2024-03-21T12:43:23.807 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 12/36 2024-03-21T12:43:23.807 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 13/36 2024-03-21T12:43:23.807 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ply-3.11-14.el9.noarch 14/36 2024-03-21T12:43:23.807 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 15/36 2024-03-21T12:43:23.807 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 16/36 2024-03-21T12:43:23.808 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 17/36 2024-03-21T12:43:23.808 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-03-21T12:43:23.808 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-03-21T12:43:23.808 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-03-21T12:43:23.808 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-03-21T12:43:23.808 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-03-21T12:43:23.808 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-03-21T12:43:23.808 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-03-21T12:43:23.808 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-03-21T12:43:23.808 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-03-21T12:43:23.808 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-03-21T12:43:23.809 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-03-21T12:43:23.809 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-03-21T12:43:23.809 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-03-21T12:43:23.810 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-03-21T12:43:23.810 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-03-21T12:43:23.810 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-03-21T12:43:23.810 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-03-21T12:43:23.810 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-03-21T12:43:24.816 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-21T12:43:24.925 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-03-21T12:43:24.925 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:24.925 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-03-21T12:43:24.925 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:43:24.925 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:43:24.925 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:43:24.925 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T12:43:24.925 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:43:24.925 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:43:24.925 INFO:teuthology.orchestra.run.smithi043.stdout: lua-5.4.4-4.el9.x86_64 2024-03-21T12:43:24.925 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-21T12:43:24.926 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-21T12:43:24.926 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-21T12:43:24.926 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-21T12:43:24.926 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-21T12:43:24.926 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-21T12:43:24.926 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-21T12:43:24.926 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-21T12:43:24.926 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-21T12:43:24.926 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-21T12:43:24.926 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-21T12:43:24.926 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-21T12:43:24.926 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-21T12:43:24.926 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-21T12:43:24.926 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-21T12:43:24.926 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-21T12:43:24.926 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-21T12:43:24.927 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply-3.11-14.el9.noarch 2024-03-21T12:43:24.927 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-21T12:43:24.927 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-21T12:43:24.927 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-21T12:43:24.927 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-21T12:43:24.927 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-21T12:43:24.927 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-21T12:43:24.927 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-21T12:43:24.927 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-21T12:43:24.927 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-21T12:43:24.927 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-21T12:43:24.927 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-21T12:43:24.927 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:24.927 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:43:25.265 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-base 2024-03-21T12:43:25.841 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:58 ago on Thu 21 Mar 2024 12:42:27 PM UTC. 2024-03-21T12:43:25.855 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-21T12:43:25.889 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2267.g960fb 12/12 2024-03-21T12:43:25.942 INFO:teuthology.orchestra.run.smithi043.stdout:Package ceph-base-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T12:43:25.996 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:43:25.997 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-03-21T12:43:25.997 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:43:26.072 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install cephadm 2024-03-21T12:43:26.643 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:59 ago on Thu 21 Mar 2024 12:42:27 PM UTC. 2024-03-21T12:43:26.797 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:43:26.798 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:43:26.798 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-03-21T12:43:26.798 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:43:26.799 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-03-21T12:43:26.799 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 750 k 2024-03-21T12:43:26.799 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:26.799 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-03-21T12:43:26.799 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:43:26.799 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-03-21T12:43:26.799 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:26.799 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 750 k 2024-03-21T12:43:26.799 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 757 k 2024-03-21T12:43:26.799 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-03-21T12:43:27.087 INFO:teuthology.orchestra.run.smithi043.stdout:cephadm-19.0.0-2267.g960fbbe4.el9.noarch.rpm 2.5 MB/s | 750 kB 00:00 2024-03-21T12:43:27.087 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:43:27.088 INFO:teuthology.orchestra.run.smithi043.stdout:Total 2.5 MB/s | 750 kB 00:00 2024-03-21T12:43:27.088 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-03-21T12:43:27.093 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-03-21T12:43:27.093 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-03-21T12:43:27.099 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-03-21T12:43:27.099 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-03-21T12:43:27.177 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-03-21T12:43:27.603 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 1/1 2024-03-21T12:43:27.693 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 1/1 2024-03-21T12:43:28.080 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 1/1 2024-03-21T12:43:28.375 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 1/1 2024-03-21T12:43:28.375 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:28.375 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-03-21T12:43:28.375 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T12:43:28.375 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:28.375 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:43:28.519 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2267.g960fb 12/12 2024-03-21T12:43:28.520 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2267.g960fb 1/12 2024-03-21T12:43:28.520 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-21T12:43:28.520 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-21T12:43:28.520 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-21T12:43:28.520 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-21T12:43:28.520 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-21T12:43:28.520 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-21T12:43:28.520 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-21T12:43:28.520 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-21T12:43:28.520 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-21T12:43:28.520 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-21T12:43:28.587 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-immutable-object-cache 2024-03-21T12:43:28.973 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-21T12:43:28.973 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:28.973 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-03-21T12:43:28.973 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T12:43:28.973 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-21T12:43:28.973 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-21T12:43:28.973 INFO:teuthology.orchestra.run.smithi155.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-21T12:43:28.973 INFO:teuthology.orchestra.run.smithi155.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-21T12:43:28.974 INFO:teuthology.orchestra.run.smithi155.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-21T12:43:28.974 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-21T12:43:28.974 INFO:teuthology.orchestra.run.smithi155.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-21T12:43:28.974 INFO:teuthology.orchestra.run.smithi155.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-21T12:43:28.974 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-21T12:43:28.974 INFO:teuthology.orchestra.run.smithi155.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-21T12:43:28.974 INFO:teuthology.orchestra.run.smithi155.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-21T12:43:28.974 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:28.974 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:43:29.164 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:02 ago on Thu 21 Mar 2024 12:42:27 PM UTC. 2024-03-21T12:43:29.320 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:43:29.321 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:43:29.321 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-03-21T12:43:29.321 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:43:29.321 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-03-21T12:43:29.321 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 145 k 2024-03-21T12:43:29.321 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:29.321 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-03-21T12:43:29.321 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:43:29.322 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-03-21T12:43:29.322 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:29.322 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 145 k 2024-03-21T12:43:29.322 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 436 k 2024-03-21T12:43:29.322 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-03-21T12:43:29.385 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-mgr-rook 2024-03-21T12:43:29.551 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-immutable-object-cache-19.0.0-2267.g960fbb 633 kB/s | 145 kB 00:00 2024-03-21T12:43:29.552 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:43:29.552 INFO:teuthology.orchestra.run.smithi043.stdout:Total 629 kB/s | 145 kB 00:00 2024-03-21T12:43:29.552 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-03-21T12:43:29.560 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-03-21T12:43:29.560 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-03-21T12:43:29.615 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-03-21T12:43:29.615 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-03-21T12:43:29.759 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-03-21T12:43:29.844 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2267.g960fbbe4. 1/1 2024-03-21T12:43:29.868 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2267.g960fbbe4. 1/1 2024-03-21T12:43:29.868 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T12:43:29.868 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-21T12:43:29.868 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:29.963 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:25 ago on Thu 21 Mar 2024 12:42:04 PM UTC. 2024-03-21T12:43:30.114 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:43:30.115 INFO:teuthology.orchestra.run.smithi155.stdout:======================================================================================== 2024-03-21T12:43:30.115 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-03-21T12:43:30.115 INFO:teuthology.orchestra.run.smithi155.stdout:======================================================================================== 2024-03-21T12:43:30.115 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-03-21T12:43:30.115 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 49 k 2024-03-21T12:43:30.115 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-03-21T12:43:30.115 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-21T12:43:30.115 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-21T12:43:30.115 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-21T12:43:30.115 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-21T12:43:30.116 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-21T12:43:30.116 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-21T12:43:30.116 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-21T12:43:30.116 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-21T12:43:30.116 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-21T12:43:30.116 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-21T12:43:30.116 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-21T12:43:30.116 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-21T12:43:30.116 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:30.116 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-03-21T12:43:30.116 INFO:teuthology.orchestra.run.smithi155.stdout:======================================================================================== 2024-03-21T12:43:30.116 INFO:teuthology.orchestra.run.smithi155.stdout:Install 13 Packages 2024-03-21T12:43:30.116 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:30.117 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 2.0 M 2024-03-21T12:43:30.117 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 26 M 2024-03-21T12:43:30.117 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-03-21T12:43:30.608 INFO:teuthology.orchestra.run.smithi155.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 195 kB/s | 24 kB 00:00 2024-03-21T12:43:30.634 INFO:teuthology.orchestra.run.smithi155.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 120 kB/s | 18 kB 00:00 2024-03-21T12:43:30.692 INFO:teuthology.orchestra.run.smithi155.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.1 MB/s | 174 kB 00:00 2024-03-21T12:43:30.717 INFO:teuthology.orchestra.run.smithi155.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 132 kB 00:00 2024-03-21T12:43:30.743 INFO:teuthology.orchestra.run.smithi155.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 4.1 MB/s | 211 kB 00:00 2024-03-21T12:43:30.768 INFO:teuthology.orchestra.run.smithi155.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 852 kB/s | 43 kB 00:00 2024-03-21T12:43:30.793 INFO:teuthology.orchestra.run.smithi155.stdout:(7/13): ceph-mgr-rook-19.0.0-2267.g960fbbe4.el9 159 kB/s | 49 kB 00:00 2024-03-21T12:43:30.852 INFO:teuthology.orchestra.run.smithi155.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 180 kB/s | 15 kB 00:00 2024-03-21T12:43:30.857 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2267.g960fbbe4. 1/1 2024-03-21T12:43:30.857 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:30.858 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-03-21T12:43:30.858 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:43:30.858 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:30.858 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:43:30.877 INFO:teuthology.orchestra.run.smithi155.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 240 kB/s | 32 kB 00:00 2024-03-21T12:43:30.920 INFO:teuthology.orchestra.run.smithi155.stdout:(10/13): python3-google-auth-2.28.2-1.el9.noarc 1.6 MB/s | 203 kB 00:00 2024-03-21T12:43:30.945 INFO:teuthology.orchestra.run.smithi155.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 881 kB/s | 59 kB 00:00 2024-03-21T12:43:30.970 INFO:teuthology.orchestra.run.smithi155.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-03-21T12:43:31.020 INFO:teuthology.orchestra.run.smithi155.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.1 MB/s | 1.0 MB 00:00 2024-03-21T12:43:31.022 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:43:31.022 INFO:teuthology.orchestra.run.smithi155.stdout:Total 2.3 MB/s | 2.0 MB 00:00 2024-03-21T12:43:31.022 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-03-21T12:43:31.035 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-03-21T12:43:31.035 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-03-21T12:43:31.125 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-03-21T12:43:31.126 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-03-21T12:43:31.159 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr 2024-03-21T12:43:31.654 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-03-21T12:43:31.733 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:04 ago on Thu 21 Mar 2024 12:42:27 PM UTC. 2024-03-21T12:43:31.825 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-21T12:43:31.833 INFO:teuthology.orchestra.run.smithi043.stdout:Package ceph-mgr-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T12:43:31.888 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:43:31.888 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-21T12:43:31.889 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-03-21T12:43:31.889 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:43:31.944 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-21T12:43:31.964 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-dashboard 2024-03-21T12:43:31.980 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-21T12:43:32.022 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-21T12:43:32.125 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-21T12:43:32.225 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-21T12:43:32.274 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-21T12:43:32.536 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:05 ago on Thu 21 Mar 2024 12:42:27 PM UTC. 2024-03-21T12:43:32.692 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:43:32.693 INFO:teuthology.orchestra.run.smithi043.stdout:====================================================================================== 2024-03-21T12:43:32.693 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-03-21T12:43:32.693 INFO:teuthology.orchestra.run.smithi043.stdout:====================================================================================== 2024-03-21T12:43:32.693 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-03-21T12:43:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 3.5 M 2024-03-21T12:43:32.694 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-03-21T12:43:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-21T12:43:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 34 k 2024-03-21T12:43:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 16 k 2024-03-21T12:43:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-21T12:43:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-21T12:43:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-21T12:43:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-21T12:43:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-21T12:43:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-21T12:43:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-21T12:43:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-21T12:43:32.694 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:32.695 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-03-21T12:43:32.695 INFO:teuthology.orchestra.run.smithi043.stdout:====================================================================================== 2024-03-21T12:43:32.695 INFO:teuthology.orchestra.run.smithi043.stdout:Install 12 Packages 2024-03-21T12:43:32.695 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:32.695 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 8.6 M 2024-03-21T12:43:32.695 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 103 M 2024-03-21T12:43:32.695 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-03-21T12:43:33.006 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-21T12:43:33.054 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-21T12:43:33.088 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-21T12:43:33.127 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-21T12:43:33.150 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-rook-2:19.0.0-2267.g960fbbe4.el9.noarch 13/13 2024-03-21T12:43:33.280 INFO:teuthology.orchestra.run.smithi043.stdout:(1/12): ceph-grafana-dashboards-19.0.0-2267.g96 144 kB/s | 34 kB 00:00 2024-03-21T12:43:33.413 INFO:teuthology.orchestra.run.smithi043.stdout:(2/12): ceph-prometheus-alerts-19.0.0-2267.g960 45 kB/s | 16 kB 00:00 2024-03-21T12:43:33.638 INFO:teuthology.orchestra.run.smithi043.stdout:(3/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.8 MB/s | 1.0 MB 00:00 2024-03-21T12:43:33.664 INFO:teuthology.orchestra.run.smithi043.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 951 kB/s | 238 kB 00:00 2024-03-21T12:43:34.016 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2267.g960fbbe4.el9.noarch 13/13 2024-03-21T12:43:34.017 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2267.g960fbbe4.el9.noarch 1/13 2024-03-21T12:43:34.017 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-21T12:43:34.017 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-21T12:43:34.017 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-21T12:43:34.017 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-21T12:43:34.017 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-21T12:43:34.017 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-21T12:43:34.017 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-21T12:43:34.017 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-21T12:43:34.017 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-21T12:43:34.017 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-21T12:43:34.017 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-21T12:43:34.047 INFO:teuthology.orchestra.run.smithi043.stdout:(5/12): protobuf-compiler-3.14.0-13.el9.x86_64. 2.1 MB/s | 863 kB 00:00 2024-03-21T12:43:34.306 INFO:teuthology.orchestra.run.smithi043.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 854 kB/s | 548 kB 00:00 2024-03-21T12:43:34.331 INFO:teuthology.orchestra.run.smithi043.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 68 kB/s | 19 kB 00:00 2024-03-21T12:43:34.373 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-21T12:43:34.374 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:34.374 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-03-21T12:43:34.374 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-rook-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T12:43:34.374 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-21T12:43:34.374 INFO:teuthology.orchestra.run.smithi155.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-21T12:43:34.374 INFO:teuthology.orchestra.run.smithi155.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-21T12:43:34.374 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-21T12:43:34.374 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-21T12:43:34.374 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-21T12:43:34.374 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-21T12:43:34.374 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-21T12:43:34.375 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-21T12:43:34.375 INFO:teuthology.orchestra.run.smithi155.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-21T12:43:34.375 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-21T12:43:34.375 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-21T12:43:34.375 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:34.375 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:43:34.581 INFO:teuthology.orchestra.run.smithi043.stdout:(8/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 7.4 MB/s | 2.0 MB 00:00 2024-03-21T12:43:34.598 INFO:teuthology.orchestra.run.smithi043.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 541 kB/s | 144 kB 00:00 2024-03-21T12:43:34.626 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-mgr-cephadm 2024-03-21T12:43:34.682 INFO:teuthology.orchestra.run.smithi043.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 307 kB/s | 31 kB 00:00 2024-03-21T12:43:34.791 INFO:teuthology.orchestra.run.smithi043.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 983 kB/s | 188 kB 00:00 2024-03-21T12:43:34.966 INFO:teuthology.orchestra.run.smithi043.stdout:(12/12): ceph-mgr-dashboard-19.0.0-2267.g960fbb 1.8 MB/s | 3.5 MB 00:01 2024-03-21T12:43:34.969 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:43:34.969 INFO:teuthology.orchestra.run.smithi043.stdout:Total 3.8 MB/s | 8.6 MB 00:02 2024-03-21T12:43:34.989 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-03-21T12:43:35.009 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-03-21T12:43:35.010 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-03-21T12:43:35.193 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:31 ago on Thu 21 Mar 2024 12:42:04 PM UTC. 2024-03-21T12:43:35.217 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-03-21T12:43:35.217 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-03-21T12:43:35.345 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:43:35.346 INFO:teuthology.orchestra.run.smithi155.stdout:======================================================================================== 2024-03-21T12:43:35.346 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-03-21T12:43:35.346 INFO:teuthology.orchestra.run.smithi155.stdout:======================================================================================== 2024-03-21T12:43:35.346 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-03-21T12:43:35.346 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 140 k 2024-03-21T12:43:35.346 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-03-21T12:43:35.346 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-21T12:43:35.346 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-21T12:43:35.346 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-21T12:43:35.346 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-21T12:43:35.346 INFO:teuthology.orchestra.run.smithi155.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-21T12:43:35.347 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:35.347 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-03-21T12:43:35.347 INFO:teuthology.orchestra.run.smithi155.stdout:======================================================================================== 2024-03-21T12:43:35.347 INFO:teuthology.orchestra.run.smithi155.stdout:Install 6 Packages 2024-03-21T12:43:35.347 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:35.347 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 6.8 M 2024-03-21T12:43:35.347 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 33 M 2024-03-21T12:43:35.348 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-03-21T12:43:35.875 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-03-21T12:43:35.950 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-03-21T12:43:35.956 INFO:teuthology.orchestra.run.smithi155.stdout:(1/6): ceph-mgr-cephadm-19.0.0-2267.g960fbbe4.e 705 kB/s | 140 kB 00:00 2024-03-21T12:43:35.985 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-03-21T12:43:36.033 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-03-21T12:43:36.071 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-03-21T12:43:36.196 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-03-21T12:43:36.333 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-03-21T12:43:36.357 INFO:teuthology.orchestra.run.smithi155.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 9.7 MB/s | 5.8 MB 00:00 2024-03-21T12:43:36.426 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-03-21T12:43:36.601 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-03-21T12:43:36.637 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-03-21T12:43:36.669 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2267.g960fbbe4.el9 10/12 2024-03-21T12:43:36.740 INFO:teuthology.orchestra.run.smithi155.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 151 kB/s | 58 kB 00:00 2024-03-21T12:43:36.865 INFO:teuthology.orchestra.run.smithi155.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 408 kB/s | 51 kB 00:00 2024-03-21T12:43:36.891 INFO:teuthology.orchestra.run.smithi155.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 587 kB/s | 548 kB 00:00 2024-03-21T12:43:36.966 INFO:teuthology.orchestra.run.smithi155.stdout:(6/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 205 kB/s | 248 kB 00:01 2024-03-21T12:43:36.968 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:43:36.968 INFO:teuthology.orchestra.run.smithi155.stdout:Total 4.2 MB/s | 6.8 MB 00:01 2024-03-21T12:43:36.983 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-03-21T12:43:36.994 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-03-21T12:43:36.994 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-03-21T12:43:37.085 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-03-21T12:43:37.085 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-03-21T12:43:37.383 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-03-21T12:43:37.556 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-21T12:43:37.609 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-21T12:43:37.772 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2267.g960fbbe4.el 11/12 2024-03-21T12:43:37.802 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2267.g960fbbe4.el9.noa 12/12 2024-03-21T12:43:38.138 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-21T12:43:38.221 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-21T12:43:38.283 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-21T12:43:38.313 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 6/6 2024-03-21T12:43:39.093 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2267.g960fbbe4.el9.noa 12/12 2024-03-21T12:43:39.093 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2267.g960fbbe4.el 1/12 2024-03-21T12:43:39.093 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2267.g960fbbe4.el9.noa 2/12 2024-03-21T12:43:39.094 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2267.g960fbbe4.el9 3/12 2024-03-21T12:43:39.094 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-03-21T12:43:39.094 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-03-21T12:43:39.094 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-03-21T12:43:39.094 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-03-21T12:43:39.094 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-03-21T12:43:39.094 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-03-21T12:43:39.094 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-03-21T12:43:39.094 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-03-21T12:43:39.128 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 6/6 2024-03-21T12:43:39.128 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 1/6 2024-03-21T12:43:39.128 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-21T12:43:39.128 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-21T12:43:39.128 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-21T12:43:39.128 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-21T12:43:39.423 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-21T12:43:39.423 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:39.423 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-03-21T12:43:39.423 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T12:43:39.423 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-21T12:43:39.423 INFO:teuthology.orchestra.run.smithi155.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-21T12:43:39.423 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-21T12:43:39.423 INFO:teuthology.orchestra.run.smithi155.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-21T12:43:39.424 INFO:teuthology.orchestra.run.smithi155.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-21T12:43:39.424 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:39.424 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:43:39.449 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-03-21T12:43:39.449 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:39.449 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-03-21T12:43:39.449 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-21T12:43:39.449 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T12:43:39.449 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T12:43:39.449 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T12:43:39.449 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-21T12:43:39.449 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-21T12:43:39.449 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-21T12:43:39.449 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-21T12:43:39.449 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-21T12:43:39.449 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-21T12:43:39.449 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-21T12:43:39.450 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-21T12:43:39.450 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:39.450 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:43:39.676 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-fuse 2024-03-21T12:43:39.704 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-diskprediction-local 2024-03-21T12:43:40.237 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:36 ago on Thu 21 Mar 2024 12:42:04 PM UTC. 2024-03-21T12:43:40.280 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:13 ago on Thu 21 Mar 2024 12:42:27 PM UTC. 2024-03-21T12:43:40.388 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:43:40.389 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:43:40.389 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-03-21T12:43:40.389 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:43:40.389 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-03-21T12:43:40.389 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 890 k 2024-03-21T12:43:40.389 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-03-21T12:43:40.390 INFO:teuthology.orchestra.run.smithi155.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-21T12:43:40.390 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:40.390 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-03-21T12:43:40.390 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:43:40.390 INFO:teuthology.orchestra.run.smithi155.stdout:Install 2 Packages 2024-03-21T12:43:40.390 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:40.390 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 969 k 2024-03-21T12:43:40.390 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 2.8 M 2024-03-21T12:43:40.391 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-03-21T12:43:40.438 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:43:40.439 INFO:teuthology.orchestra.run.smithi043.stdout:============================================================================================ 2024-03-21T12:43:40.439 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-03-21T12:43:40.439 INFO:teuthology.orchestra.run.smithi043.stdout:============================================================================================ 2024-03-21T12:43:40.439 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-03-21T12:43:40.439 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 7.4 M 2024-03-21T12:43:40.439 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-03-21T12:43:40.439 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-21T12:43:40.439 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-21T12:43:40.440 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-21T12:43:40.440 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-21T12:43:40.440 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-21T12:43:40.440 INFO:teuthology.orchestra.run.smithi043.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-21T12:43:40.440 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-21T12:43:40.440 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-21T12:43:40.440 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-21T12:43:40.440 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-21T12:43:40.440 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-21T12:43:40.440 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:40.440 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-03-21T12:43:40.440 INFO:teuthology.orchestra.run.smithi043.stdout:============================================================================================ 2024-03-21T12:43:40.440 INFO:teuthology.orchestra.run.smithi043.stdout:Install 12 Packages 2024-03-21T12:43:40.440 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:40.441 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 37 M 2024-03-21T12:43:40.441 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 209 M 2024-03-21T12:43:40.441 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-03-21T12:43:40.610 INFO:teuthology.orchestra.run.smithi155.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 359 kB/s | 79 kB 00:00 2024-03-21T12:43:40.893 INFO:teuthology.orchestra.run.smithi155.stdout:(2/2): ceph-fuse-19.0.0-2267.g960fbbe4.el9.x86_ 1.7 MB/s | 890 kB 00:00 2024-03-21T12:43:40.894 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:43:40.894 INFO:teuthology.orchestra.run.smithi155.stdout:Total 1.9 MB/s | 969 kB 00:00 2024-03-21T12:43:40.895 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-03-21T12:43:40.915 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-03-21T12:43:40.915 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-03-21T12:43:40.988 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-03-21T12:43:40.988 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-03-21T12:43:41.090 INFO:teuthology.orchestra.run.smithi043.stdout:(1/12): flexiblas-3.0.4-8.el9.x86_64.rpm 160 kB/s | 32 kB 00:00 2024-03-21T12:43:41.149 INFO:teuthology.orchestra.run.smithi043.stdout:(2/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 294 kB/s | 17 kB 00:00 2024-03-21T12:43:41.164 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-03-21T12:43:41.216 INFO:teuthology.orchestra.run.smithi043.stdout:(3/12): openblas-0.3.21-2.el9.x86_64.rpm 512 kB/s | 34 kB 00:00 2024-03-21T12:43:41.321 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-21T12:43:41.350 INFO:teuthology.orchestra.run.smithi043.stdout:(4/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 6.5 MB/s | 3.0 MB 00:00 2024-03-21T12:43:41.393 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 2/2 2024-03-21T12:43:41.592 INFO:teuthology.orchestra.run.smithi043.stdout:(5/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 13 MB/s | 4.7 MB 00:00 2024-03-21T12:43:41.742 INFO:teuthology.orchestra.run.smithi043.stdout:(6/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 4.8 MB 00:00 2024-03-21T12:43:41.767 INFO:teuthology.orchestra.run.smithi043.stdout:(7/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.2 MB/s | 217 kB 00:00 2024-03-21T12:43:42.076 INFO:teuthology.orchestra.run.smithi043.stdout:(8/12): ceph-mgr-diskprediction-local-19.0.0-22 6.3 MB/s | 7.4 MB 00:01 2024-03-21T12:43:42.114 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 2/2 2024-03-21T12:43:42.114 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/2 2024-03-21T12:43:42.399 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-21T12:43:42.399 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:42.400 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-03-21T12:43:42.400 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-21T12:43:42.400 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:42.400 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:43:42.409 INFO:teuthology.orchestra.run.smithi043.stdout:(9/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-03-21T12:43:42.510 INFO:teuthology.orchestra.run.smithi043.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 454 kB/s | 195 kB 00:00 2024-03-21T12:43:42.568 INFO:teuthology.orchestra.run.smithi043.stdout:(11/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 803 kB 00:00 2024-03-21T12:43:42.593 INFO:teuthology.orchestra.run.smithi043.stdout:(12/12): python3-devel-3.9.17-2.el9.x86_64.rpm 1.3 MB/s | 245 kB 00:00 2024-03-21T12:43:42.597 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:43:42.597 INFO:teuthology.orchestra.run.smithi043.stdout:Total 17 MB/s | 37 MB 00:02 2024-03-21T12:43:42.638 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-03-21T12:43:42.663 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-03-21T12:43:42.663 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-03-21T12:43:42.715 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install ceph-volume 2024-03-21T12:43:43.029 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-03-21T12:43:43.029 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-03-21T12:43:43.296 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:39 ago on Thu 21 Mar 2024 12:42:04 PM UTC. 2024-03-21T12:43:43.446 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:43:43.447 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:43:43.447 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repository Size 2024-03-21T12:43:43.447 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:43:43.447 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-03-21T12:43:43.447 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 273 k 2024-03-21T12:43:43.448 INFO:teuthology.orchestra.run.smithi155.stdout:Installing dependencies: 2024-03-21T12:43:43.448 INFO:teuthology.orchestra.run.smithi155.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-21T12:43:43.448 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:43.448 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-03-21T12:43:43.448 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:43:43.448 INFO:teuthology.orchestra.run.smithi155.stdout:Install 2 Packages 2024-03-21T12:43:43.448 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:43.448 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 342 k 2024-03-21T12:43:43.448 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 1.5 M 2024-03-21T12:43:43.449 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-03-21T12:43:43.750 INFO:teuthology.orchestra.run.smithi155.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 228 kB/s | 69 kB 00:00 2024-03-21T12:43:43.775 INFO:teuthology.orchestra.run.smithi155.stdout:(2/2): ceph-volume-19.0.0-2267.g960fbbe4.el9.no 837 kB/s | 273 kB 00:00 2024-03-21T12:43:43.777 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:43:43.777 INFO:teuthology.orchestra.run.smithi155.stdout:Total 1.0 MB/s | 342 kB 00:00 2024-03-21T12:43:43.778 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-03-21T12:43:43.786 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-03-21T12:43:43.786 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-03-21T12:43:43.819 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-03-21T12:43:43.819 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-03-21T12:43:43.867 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-03-21T12:43:43.942 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 1/12 2024-03-21T12:43:43.969 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-03-21T12:43:43.972 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-03-21T12:43:44.074 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 3/12 2024-03-21T12:43:44.118 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 4/12 2024-03-21T12:43:44.166 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-03-21T12:43:44.203 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : ceph-volume-2:19.0.0-2267.g960fbbe4.el9.noarch 2/2 2024-03-21T12:43:44.218 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: ceph-volume-2:19.0.0-2267.g960fbbe4.el9.noarch 2/2 2024-03-21T12:43:44.219 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T12:43:44.219 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-21T12:43:44.219 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:44.301 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-21T12:43:44.333 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-21T12:43:44.420 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-03-21T12:43:44.935 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : ceph-volume-2:19.0.0-2267.g960fbbe4.el9.noarch 1/2 2024-03-21T12:43:45.109 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-03-21T12:43:45.215 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-03-21T12:43:45.264 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-03-21T12:43:45.265 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:45.265 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-03-21T12:43:45.265 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-volume-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T12:43:45.265 INFO:teuthology.orchestra.run.smithi155.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-21T12:43:45.265 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:45.265 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:43:45.626 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install librados-devel 2024-03-21T12:43:46.201 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:42 ago on Thu 21 Mar 2024 12:42:04 PM UTC. 2024-03-21T12:43:46.357 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:43:46.357 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:43:46.357 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repo Size 2024-03-21T12:43:46.357 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:43:46.357 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-03-21T12:43:46.357 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 127 k 2024-03-21T12:43:46.358 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:46.358 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-03-21T12:43:46.358 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:43:46.358 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-03-21T12:43:46.358 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:46.358 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 127 k 2024-03-21T12:43:46.358 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 456 k 2024-03-21T12:43:46.358 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-03-21T12:43:46.529 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-03-21T12:43:46.824 INFO:teuthology.orchestra.run.smithi155.stdout:librados-devel-19.0.0-2267.g960fbbe4.el9.x86_64 272 kB/s | 127 kB 00:00 2024-03-21T12:43:46.824 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:43:46.824 INFO:teuthology.orchestra.run.smithi155.stdout:Total 272 kB/s | 127 kB 00:00 2024-03-21T12:43:46.825 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-03-21T12:43:46.830 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-03-21T12:43:46.830 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-03-21T12:43:46.891 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-03-21T12:43:46.891 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-03-21T12:43:47.110 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-03-21T12:43:47.213 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : librados-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:43:47.602 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-03-21T12:43:47.632 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2267.g960fb 12/12 2024-03-21T12:43:47.648 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: librados-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:43:47.937 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : librados-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:43:47.937 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:47.937 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-03-21T12:43:47.938 INFO:teuthology.orchestra.run.smithi155.stdout: librados-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:43:47.938 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:47.938 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:43:48.184 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install libcephfs2 2024-03-21T12:43:48.761 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:44 ago on Thu 21 Mar 2024 12:42:04 PM UTC. 2024-03-21T12:43:48.862 INFO:teuthology.orchestra.run.smithi155.stdout:Package libcephfs2-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T12:43:48.913 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:43:48.915 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-03-21T12:43:48.915 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:43:48.971 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install libcephfs-devel 2024-03-21T12:43:49.534 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:45 ago on Thu 21 Mar 2024 12:42:04 PM UTC. 2024-03-21T12:43:49.683 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:43:49.684 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:43:49.684 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repo Size 2024-03-21T12:43:49.684 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:43:49.684 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-03-21T12:43:49.684 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 31 k 2024-03-21T12:43:49.684 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:49.684 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-03-21T12:43:49.684 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:43:49.684 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-03-21T12:43:49.684 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:49.685 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 31 k 2024-03-21T12:43:49.685 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 140 k 2024-03-21T12:43:49.685 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-03-21T12:43:49.986 INFO:teuthology.orchestra.run.smithi155.stdout:libcephfs-devel-19.0.0-2267.g960fbbe4.el9.x86_6 104 kB/s | 31 kB 00:00 2024-03-21T12:43:49.986 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:43:49.986 INFO:teuthology.orchestra.run.smithi155.stdout:Total 104 kB/s | 31 kB 00:00 2024-03-21T12:43:49.986 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-03-21T12:43:49.990 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-03-21T12:43:49.990 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-03-21T12:43:50.022 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-03-21T12:43:50.022 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-03-21T12:43:50.117 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-03-21T12:43:50.169 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2267.g960fb 12/12 2024-03-21T12:43:50.169 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2267.g960fb 1/12 2024-03-21T12:43:50.169 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 2/12 2024-03-21T12:43:50.169 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 3/12 2024-03-21T12:43:50.169 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 4/12 2024-03-21T12:43:50.169 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 5/12 2024-03-21T12:43:50.169 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-03-21T12:43:50.169 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 7/12 2024-03-21T12:43:50.170 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 8/12 2024-03-21T12:43:50.170 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 9/12 2024-03-21T12:43:50.170 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 10/12 2024-03-21T12:43:50.170 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 11/12 2024-03-21T12:43:50.267 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : libcephfs-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:43:50.566 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 12/12 2024-03-21T12:43:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:50.567 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-03-21T12:43:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T12:43:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-21T12:43:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-21T12:43:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-21T12:43:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-21T12:43:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-21T12:43:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-21T12:43:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-21T12:43:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-21T12:43:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-21T12:43:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-21T12:43:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-21T12:43:50.567 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:50.568 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:43:50.654 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:43:50.913 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : libcephfs-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:43:50.913 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:50.913 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-03-21T12:43:50.913 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:43:50.914 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:50.914 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:43:50.969 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-rook 2024-03-21T12:43:51.100 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install librados2 2024-03-21T12:43:51.540 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:24 ago on Thu 21 Mar 2024 12:42:27 PM UTC. 2024-03-21T12:43:51.661 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:47 ago on Thu 21 Mar 2024 12:42:04 PM UTC. 2024-03-21T12:43:51.697 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:43:51.698 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================== 2024-03-21T12:43:51.698 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-03-21T12:43:51.698 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================== 2024-03-21T12:43:51.699 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-03-21T12:43:51.699 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 49 k 2024-03-21T12:43:51.699 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-03-21T12:43:51.699 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-21T12:43:51.699 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-21T12:43:51.699 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-21T12:43:51.699 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-21T12:43:51.699 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-21T12:43:51.699 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-21T12:43:51.699 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-21T12:43:51.699 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-21T12:43:51.699 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-21T12:43:51.699 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-21T12:43:51.699 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-21T12:43:51.700 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-21T12:43:51.700 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:51.700 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-03-21T12:43:51.700 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================== 2024-03-21T12:43:51.700 INFO:teuthology.orchestra.run.smithi043.stdout:Install 13 Packages 2024-03-21T12:43:51.700 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:51.700 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 2.0 M 2024-03-21T12:43:51.700 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 26 M 2024-03-21T12:43:51.700 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-03-21T12:43:51.758 INFO:teuthology.orchestra.run.smithi155.stdout:Package librados2-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T12:43:51.811 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:43:51.812 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-03-21T12:43:51.812 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:43:51.877 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install librbd1 2024-03-21T12:43:52.256 INFO:teuthology.orchestra.run.smithi043.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 126 kB/s | 24 kB 00:00 2024-03-21T12:43:52.282 INFO:teuthology.orchestra.run.smithi043.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 83 kB/s | 18 kB 00:00 2024-03-21T12:43:52.407 INFO:teuthology.orchestra.run.smithi043.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.1 MB/s | 174 kB 00:00 2024-03-21T12:43:52.432 INFO:teuthology.orchestra.run.smithi043.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 880 kB/s | 132 kB 00:00 2024-03-21T12:43:52.436 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:48 ago on Thu 21 Mar 2024 12:42:04 PM UTC. 2024-03-21T12:43:52.491 INFO:teuthology.orchestra.run.smithi043.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.5 MB/s | 211 kB 00:00 2024-03-21T12:43:52.508 INFO:teuthology.orchestra.run.smithi043.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 567 kB/s | 43 kB 00:00 2024-03-21T12:43:52.534 INFO:teuthology.orchestra.run.smithi155.stdout:Package librbd1-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T12:43:52.550 INFO:teuthology.orchestra.run.smithi043.stdout:(7/13): ceph-mgr-rook-19.0.0-2267.g960fbbe4.el9 101 kB/s | 49 kB 00:00 2024-03-21T12:43:52.583 INFO:teuthology.orchestra.run.smithi043.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 199 kB/s | 15 kB 00:00 2024-03-21T12:43:52.586 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:43:52.587 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-03-21T12:43:52.587 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:43:52.600 INFO:teuthology.orchestra.run.smithi043.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 295 kB/s | 32 kB 00:00 2024-03-21T12:43:52.650 INFO:teuthology.orchestra.run.smithi043.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-03-21T12:43:52.652 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install python3-rados 2024-03-21T12:43:52.676 INFO:teuthology.orchestra.run.smithi043.stdout:(11/13): python3-google-auth-2.28.2-1.el9.noarc 1.6 MB/s | 203 kB 00:00 2024-03-21T12:43:52.701 INFO:teuthology.orchestra.run.smithi043.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-03-21T12:43:52.734 INFO:teuthology.orchestra.run.smithi043.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-03-21T12:43:52.737 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:43:52.774 INFO:teuthology.orchestra.run.smithi043.stdout:Total 2.0 MB/s | 2.0 MB 00:01 2024-03-21T12:43:52.775 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-03-21T12:43:52.775 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-03-21T12:43:52.775 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-03-21T12:43:52.853 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-03-21T12:43:52.854 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-03-21T12:43:53.216 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:49 ago on Thu 21 Mar 2024 12:42:04 PM UTC. 2024-03-21T12:43:53.313 INFO:teuthology.orchestra.run.smithi155.stdout:Package python3-rados-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T12:43:53.365 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:43:53.366 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-03-21T12:43:53.366 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:43:53.435 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install python3-rgw 2024-03-21T12:43:53.457 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-03-21T12:43:53.630 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-03-21T12:43:53.686 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-03-21T12:43:53.742 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-03-21T12:43:53.778 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-03-21T12:43:53.820 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-03-21T12:43:53.921 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-03-21T12:43:54.026 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:50 ago on Thu 21 Mar 2024 12:42:04 PM UTC. 2024-03-21T12:43:54.026 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 7/13 2024-03-21T12:43:54.072 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-03-21T12:43:54.133 INFO:teuthology.orchestra.run.smithi155.stdout:Package python3-rgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T12:43:54.185 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:43:54.186 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-03-21T12:43:54.187 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:43:54.255 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install python3-cephfs 2024-03-21T12:43:54.815 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:50 ago on Thu 21 Mar 2024 12:42:04 PM UTC. 2024-03-21T12:43:54.827 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-03-21T12:43:54.868 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-03-21T12:43:54.902 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-03-21T12:43:54.911 INFO:teuthology.orchestra.run.smithi155.stdout:Package python3-cephfs-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T12:43:54.959 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-03-21T12:43:54.963 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:43:54.964 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-03-21T12:43:54.964 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:43:54.988 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-rook-2:19.0.0-2267.g960fbbe4.el9.noarch 13/13 2024-03-21T12:43:55.027 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install python3-rbd 2024-03-21T12:43:55.597 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:51 ago on Thu 21 Mar 2024 12:42:04 PM UTC. 2024-03-21T12:43:55.694 INFO:teuthology.orchestra.run.smithi155.stdout:Package python3-rbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T12:43:55.747 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:43:55.749 INFO:teuthology.orchestra.run.smithi155.stdout:Nothing to do. 2024-03-21T12:43:55.749 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:43:55.763 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2267.g960fbbe4.el9.noarch 13/13 2024-03-21T12:43:55.763 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2267.g960fbbe4.el9.noarch 1/13 2024-03-21T12:43:55.763 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-03-21T12:43:55.763 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-03-21T12:43:55.763 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-03-21T12:43:55.763 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-03-21T12:43:55.763 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-03-21T12:43:55.763 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-03-21T12:43:55.763 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-03-21T12:43:55.764 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-03-21T12:43:55.764 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 10/13 2024-03-21T12:43:55.764 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-03-21T12:43:55.764 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-03-21T12:43:55.823 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install rbd-fuse 2024-03-21T12:43:56.131 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-03-21T12:43:56.131 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:56.131 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-03-21T12:43:56.131 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T12:43:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-21T12:43:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-21T12:43:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-21T12:43:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-21T12:43:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-21T12:43:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-21T12:43:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-21T12:43:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-21T12:43:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-21T12:43:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-21T12:43:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-21T12:43:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-21T12:43:56.132 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:56.132 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:43:56.351 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-mgr-cephadm 2024-03-21T12:43:56.381 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:52 ago on Thu 21 Mar 2024 12:42:04 PM UTC. 2024-03-21T12:43:56.531 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:43:56.532 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:43:56.532 INFO:teuthology.orchestra.run.smithi155.stdout: Package Architecture Version Repository Size 2024-03-21T12:43:56.532 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:43:56.532 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-03-21T12:43:56.532 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 85 k 2024-03-21T12:43:56.532 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:56.532 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-03-21T12:43:56.533 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:43:56.533 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-03-21T12:43:56.533 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:56.533 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 85 k 2024-03-21T12:43:56.533 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 231 k 2024-03-21T12:43:56.533 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-03-21T12:43:56.746 INFO:teuthology.orchestra.run.smithi155.stdout:rbd-fuse-19.0.0-2267.g960fbbe4.el9.x86_64.rpm 399 kB/s | 85 kB 00:00 2024-03-21T12:43:56.747 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:43:56.747 INFO:teuthology.orchestra.run.smithi155.stdout:Total 397 kB/s | 85 kB 00:00 2024-03-21T12:43:56.747 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-03-21T12:43:56.755 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-03-21T12:43:56.755 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-03-21T12:43:56.809 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-03-21T12:43:56.810 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-03-21T12:43:56.924 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:29 ago on Thu 21 Mar 2024 12:42:27 PM UTC. 2024-03-21T12:43:56.948 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-03-21T12:43:57.045 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:43:57.080 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:43:57.081 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================== 2024-03-21T12:43:57.081 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-03-21T12:43:57.081 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================== 2024-03-21T12:43:57.081 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-03-21T12:43:57.081 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 140 k 2024-03-21T12:43:57.081 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-03-21T12:43:57.081 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-21T12:43:57.081 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-21T12:43:57.082 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-21T12:43:57.082 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-21T12:43:57.082 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-21T12:43:57.082 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:57.082 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-03-21T12:43:57.082 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================== 2024-03-21T12:43:57.082 INFO:teuthology.orchestra.run.smithi043.stdout:Install 6 Packages 2024-03-21T12:43:57.082 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:43:57.083 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 6.8 M 2024-03-21T12:43:57.083 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 33 M 2024-03-21T12:43:57.083 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-03-21T12:43:57.496 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:43:57.527 INFO:teuthology.orchestra.run.smithi043.stdout:(1/6): ceph-mgr-cephadm-19.0.0-2267.g960fbbe4.e 647 kB/s | 140 kB 00:00 2024-03-21T12:43:57.619 INFO:teuthology.orchestra.run.smithi043.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 804 kB/s | 248 kB 00:00 2024-03-21T12:43:57.811 INFO:teuthology.orchestra.run.smithi043.stdout:(3/6): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 5.8 MB 00:00 2024-03-21T12:43:57.815 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:43:57.815 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:57.815 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-03-21T12:43:57.816 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:43:57.816 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:57.816 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:43:57.920 INFO:teuthology.orchestra.run.smithi043.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 193 kB/s | 58 kB 00:00 2024-03-21T12:43:57.986 INFO:teuthology.orchestra.run.smithi043.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.2 MB/s | 548 kB 00:00 2024-03-21T12:43:58.036 INFO:teuthology.orchestra.run.smithi043.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 227 kB/s | 51 kB 00:00 2024-03-21T12:43:58.040 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:43:58.040 INFO:teuthology.orchestra.run.smithi043.stdout:Total 7.1 MB/s | 6.8 MB 00:00 2024-03-21T12:43:58.053 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-03-21T12:43:58.063 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-03-21T12:43:58.063 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-03-21T12:43:58.067 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install rbd-mirror 2024-03-21T12:43:58.156 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-03-21T12:43:58.156 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-03-21T12:43:58.469 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-03-21T12:43:58.652 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-03-21T12:43:58.663 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:01:54 ago on Thu 21 Mar 2024 12:42:04 PM UTC. 2024-03-21T12:43:58.697 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-03-21T12:43:58.818 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:43:58.819 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:43:58.819 INFO:teuthology.orchestra.run.smithi155.stdout: Package Arch Version Repo Size 2024-03-21T12:43:58.819 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:43:58.819 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-03-21T12:43:58.819 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 3.1 M 2024-03-21T12:43:58.820 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:58.820 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-03-21T12:43:58.820 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:43:58.820 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-03-21T12:43:58.820 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:43:58.820 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 3.1 M 2024-03-21T12:43:58.820 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 13 M 2024-03-21T12:43:58.820 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-03-21T12:43:59.245 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-03-21T12:43:59.318 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-03-21T12:43:59.396 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-03-21T12:43:59.425 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 6/6 2024-03-21T12:44:00.256 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 6/6 2024-03-21T12:44:00.256 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 1/6 2024-03-21T12:44:00.256 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-03-21T12:44:00.257 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-03-21T12:44:00.257 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-03-21T12:44:00.257 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-03-21T12:44:00.572 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-03-21T12:44:00.572 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:00.572 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-03-21T12:44:00.572 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T12:44:00.572 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-21T12:44:00.572 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-21T12:44:00.572 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-21T12:44:00.572 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-21T12:44:00.572 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-21T12:44:00.572 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:00.572 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:44:00.810 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-fuse 2024-03-21T12:44:01.377 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:34 ago on Thu 21 Mar 2024 12:42:27 PM UTC. 2024-03-21T12:44:01.534 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:44:01.535 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:44:01.535 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-03-21T12:44:01.535 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:44:01.535 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-03-21T12:44:01.535 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 890 k 2024-03-21T12:44:01.535 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-03-21T12:44:01.536 INFO:teuthology.orchestra.run.smithi043.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-21T12:44:01.536 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:01.536 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-03-21T12:44:01.536 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:44:01.536 INFO:teuthology.orchestra.run.smithi043.stdout:Install 2 Packages 2024-03-21T12:44:01.536 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:01.537 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 969 k 2024-03-21T12:44:01.537 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 2.8 M 2024-03-21T12:44:01.537 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-03-21T12:44:01.584 INFO:teuthology.orchestra.run.smithi155.stdout:rbd-mirror-19.0.0-2267.g960fbbe4.el9.x86_64.rpm 1.1 MB/s | 3.1 MB 00:02 2024-03-21T12:44:01.585 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:44:01.585 INFO:teuthology.orchestra.run.smithi155.stdout:Total 1.1 MB/s | 3.1 MB 00:02 2024-03-21T12:44:01.585 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-03-21T12:44:01.596 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-03-21T12:44:01.596 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-03-21T12:44:01.672 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-03-21T12:44:01.673 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-03-21T12:44:01.679 INFO:teuthology.orchestra.run.smithi043.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 551 kB/s | 79 kB 00:00 2024-03-21T12:44:02.091 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-03-21T12:44:02.160 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-mirror-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:44:02.185 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:44:02.185 INFO:teuthology.orchestra.run.smithi155.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T12:44:02.185 INFO:teuthology.orchestra.run.smithi155.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-21T12:44:02.185 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T12:44:02.185 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T12:44:02.185 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:44:02.629 INFO:teuthology.orchestra.run.smithi043.stdout:(2/2): ceph-fuse-19.0.0-2267.g960fbbe4.el9.x86_ 814 kB/s | 890 kB 00:01 2024-03-21T12:44:02.631 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:44:02.631 INFO:teuthology.orchestra.run.smithi043.stdout:Total 884 kB/s | 969 kB 00:01 2024-03-21T12:44:02.632 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-03-21T12:44:02.656 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-03-21T12:44:02.656 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-03-21T12:44:02.736 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-03-21T12:44:02.736 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-03-21T12:44:02.924 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-03-21T12:44:03.067 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-03-21T12:44:03.143 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 2/2 2024-03-21T12:44:03.319 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-mirror-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:44:03.319 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:44:03.319 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-03-21T12:44:03.319 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-mirror-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:44:03.319 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:44:03.319 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:44:03.600 DEBUG:teuthology.orchestra.run.smithi155:> sudo yum -y install rbd-nbd 2024-03-21T12:44:03.849 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 2/2 2024-03-21T12:44:03.849 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/2 2024-03-21T12:44:04.149 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-03-21T12:44:04.149 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:04.149 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-03-21T12:44:04.149 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-03-21T12:44:04.149 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:04.149 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:44:04.173 INFO:teuthology.orchestra.run.smithi155.stdout:Last metadata expiration check: 0:02:00 ago on Thu 21 Mar 2024 12:42:04 PM UTC. 2024-03-21T12:44:04.323 INFO:teuthology.orchestra.run.smithi155.stdout:Dependencies resolved. 2024-03-21T12:44:04.323 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:44:04.323 INFO:teuthology.orchestra.run.smithi155.stdout: Package Architecture Version Repository Size 2024-03-21T12:44:04.323 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:44:04.324 INFO:teuthology.orchestra.run.smithi155.stdout:Installing: 2024-03-21T12:44:04.324 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 170 k 2024-03-21T12:44:04.324 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:44:04.324 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction Summary 2024-03-21T12:44:04.324 INFO:teuthology.orchestra.run.smithi155.stdout:================================================================================ 2024-03-21T12:44:04.324 INFO:teuthology.orchestra.run.smithi155.stdout:Install 1 Package 2024-03-21T12:44:04.324 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:44:04.324 INFO:teuthology.orchestra.run.smithi155.stdout:Total download size: 170 k 2024-03-21T12:44:04.325 INFO:teuthology.orchestra.run.smithi155.stdout:Installed size: 490 k 2024-03-21T12:44:04.325 INFO:teuthology.orchestra.run.smithi155.stdout:Downloading Packages: 2024-03-21T12:44:04.455 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-volume 2024-03-21T12:44:04.555 INFO:teuthology.orchestra.run.smithi155.stdout:rbd-nbd-19.0.0-2267.g960fbbe4.el9.x86_64.rpm 741 kB/s | 170 kB 00:00 2024-03-21T12:44:04.555 INFO:teuthology.orchestra.run.smithi155.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:44:04.556 INFO:teuthology.orchestra.run.smithi155.stdout:Total 737 kB/s | 170 kB 00:00 2024-03-21T12:44:04.556 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction check 2024-03-21T12:44:04.564 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction check succeeded. 2024-03-21T12:44:04.564 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction test 2024-03-21T12:44:04.619 INFO:teuthology.orchestra.run.smithi155.stdout:Transaction test succeeded. 2024-03-21T12:44:04.619 INFO:teuthology.orchestra.run.smithi155.stdout:Running transaction 2024-03-21T12:44:04.774 INFO:teuthology.orchestra.run.smithi155.stdout: Preparing : 1/1 2024-03-21T12:44:04.887 INFO:teuthology.orchestra.run.smithi155.stdout: Installing : rbd-nbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:44:05.039 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:38 ago on Thu 21 Mar 2024 12:42:27 PM UTC. 2024-03-21T12:44:05.197 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:44:05.198 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:44:05.198 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-03-21T12:44:05.198 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:44:05.198 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-03-21T12:44:05.198 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume noarch 2:19.0.0-2267.g960fbbe4.el9 ceph-noarch 273 k 2024-03-21T12:44:05.198 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-03-21T12:44:05.198 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-21T12:44:05.198 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:05.199 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-03-21T12:44:05.199 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:44:05.199 INFO:teuthology.orchestra.run.smithi043.stdout:Install 2 Packages 2024-03-21T12:44:05.199 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:05.199 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 342 k 2024-03-21T12:44:05.199 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 1.5 M 2024-03-21T12:44:05.199 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-03-21T12:44:05.336 INFO:teuthology.orchestra.run.smithi043.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 504 kB/s | 69 kB 00:00 2024-03-21T12:44:05.346 INFO:teuthology.orchestra.run.smithi155.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:44:05.586 INFO:teuthology.orchestra.run.smithi043.stdout:(2/2): ceph-volume-19.0.0-2267.g960fbbe4.el9.no 707 kB/s | 273 kB 00:00 2024-03-21T12:44:05.587 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:44:05.588 INFO:teuthology.orchestra.run.smithi043.stdout:Total 879 kB/s | 342 kB 00:00 2024-03-21T12:44:05.588 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-03-21T12:44:05.597 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-03-21T12:44:05.597 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-03-21T12:44:05.627 INFO:teuthology.orchestra.run.smithi155.stdout: Verifying : rbd-nbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:44:05.628 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:44:05.628 INFO:teuthology.orchestra.run.smithi155.stdout:Installed: 2024-03-21T12:44:05.628 INFO:teuthology.orchestra.run.smithi155.stdout: rbd-nbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:44:05.628 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:44:05.628 INFO:teuthology.orchestra.run.smithi155.stdout:Complete! 2024-03-21T12:44:05.631 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-03-21T12:44:05.632 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-03-21T12:44:05.782 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-03-21T12:44:05.859 DEBUG:teuthology.parallel:result is None 2024-03-21T12:44:05.976 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-03-21T12:44:06.014 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-volume-2:19.0.0-2267.g960fbbe4.el9.noarch 2/2 2024-03-21T12:44:06.030 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-volume-2:19.0.0-2267.g960fbbe4.el9.noarch 2/2 2024-03-21T12:44:06.030 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T12:44:06.030 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-21T12:44:06.030 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:06.810 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-volume-2:19.0.0-2267.g960fbbe4.el9.noarch 1/2 2024-03-21T12:44:07.161 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-03-21T12:44:07.161 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:07.161 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-03-21T12:44:07.161 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume-2:19.0.0-2267.g960fbbe4.el9.noarch 2024-03-21T12:44:07.162 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-21T12:44:07.162 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:07.162 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:44:07.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install librados-devel 2024-03-21T12:44:08.058 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:41 ago on Thu 21 Mar 2024 12:42:27 PM UTC. 2024-03-21T12:44:08.215 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:44:08.216 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:44:08.216 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-03-21T12:44:08.216 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:44:08.216 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-03-21T12:44:08.216 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 127 k 2024-03-21T12:44:08.216 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:08.216 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-03-21T12:44:08.216 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:44:08.217 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-03-21T12:44:08.217 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:08.217 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 127 k 2024-03-21T12:44:08.217 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 456 k 2024-03-21T12:44:08.217 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-03-21T12:44:08.443 INFO:teuthology.orchestra.run.smithi043.stdout:librados-devel-19.0.0-2267.g960fbbe4.el9.x86_64 563 kB/s | 127 kB 00:00 2024-03-21T12:44:08.443 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:44:08.443 INFO:teuthology.orchestra.run.smithi043.stdout:Total 559 kB/s | 127 kB 00:00 2024-03-21T12:44:08.443 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-03-21T12:44:08.449 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-03-21T12:44:08.449 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-03-21T12:44:08.511 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-03-21T12:44:08.512 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-03-21T12:44:08.725 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-03-21T12:44:08.835 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librados-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:44:09.271 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:44:09.595 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:44:09.595 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:09.595 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-03-21T12:44:09.595 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:44:09.595 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:09.595 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:44:09.795 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install libcephfs2 2024-03-21T12:44:10.385 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:43 ago on Thu 21 Mar 2024 12:42:27 PM UTC. 2024-03-21T12:44:10.486 INFO:teuthology.orchestra.run.smithi043.stdout:Package libcephfs2-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T12:44:10.541 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:44:10.543 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-03-21T12:44:10.543 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:44:10.595 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install libcephfs-devel 2024-03-21T12:44:11.173 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:44 ago on Thu 21 Mar 2024 12:42:27 PM UTC. 2024-03-21T12:44:11.330 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:44:11.330 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:44:11.330 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-03-21T12:44:11.331 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:44:11.331 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-03-21T12:44:11.331 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 31 k 2024-03-21T12:44:11.331 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:11.331 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-03-21T12:44:11.331 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:44:11.331 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-03-21T12:44:11.331 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:11.331 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 31 k 2024-03-21T12:44:11.331 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 140 k 2024-03-21T12:44:11.331 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-03-21T12:44:11.496 INFO:teuthology.orchestra.run.smithi043.stdout:libcephfs-devel-19.0.0-2267.g960fbbe4.el9.x86_6 190 kB/s | 31 kB 00:00 2024-03-21T12:44:11.496 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:44:11.497 INFO:teuthology.orchestra.run.smithi043.stdout:Total 188 kB/s | 31 kB 00:00 2024-03-21T12:44:11.497 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-03-21T12:44:11.501 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-03-21T12:44:11.501 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-03-21T12:44:11.518 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-03-21T12:44:11.518 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-03-21T12:44:11.608 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-03-21T12:44:11.738 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:44:12.147 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:44:12.411 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:44:12.411 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:12.411 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-03-21T12:44:12.412 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:44:12.412 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:12.412 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:44:12.581 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install librados2 2024-03-21T12:44:13.156 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:46 ago on Thu 21 Mar 2024 12:42:27 PM UTC. 2024-03-21T12:44:13.256 INFO:teuthology.orchestra.run.smithi043.stdout:Package librados2-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T12:44:13.311 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:44:13.312 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-03-21T12:44:13.312 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:44:13.371 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install librbd1 2024-03-21T12:44:13.943 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:46 ago on Thu 21 Mar 2024 12:42:27 PM UTC. 2024-03-21T12:44:14.044 INFO:teuthology.orchestra.run.smithi043.stdout:Package librbd1-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T12:44:14.099 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:44:14.101 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-03-21T12:44:14.101 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:44:14.162 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-rados 2024-03-21T12:44:14.739 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:47 ago on Thu 21 Mar 2024 12:42:27 PM UTC. 2024-03-21T12:44:14.839 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-rados-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T12:44:14.893 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:44:14.894 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-03-21T12:44:14.895 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:44:14.953 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-rgw 2024-03-21T12:44:15.532 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:48 ago on Thu 21 Mar 2024 12:42:27 PM UTC. 2024-03-21T12:44:15.632 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-rgw-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T12:44:15.689 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:44:15.690 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-03-21T12:44:15.690 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:44:15.754 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-cephfs 2024-03-21T12:44:16.326 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:49 ago on Thu 21 Mar 2024 12:42:27 PM UTC. 2024-03-21T12:44:16.426 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-cephfs-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T12:44:16.483 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:44:16.484 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-03-21T12:44:16.484 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:44:16.547 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install python3-rbd 2024-03-21T12:44:17.123 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:50 ago on Thu 21 Mar 2024 12:42:27 PM UTC. 2024-03-21T12:44:17.223 INFO:teuthology.orchestra.run.smithi043.stdout:Package python3-rbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 is already installed. 2024-03-21T12:44:17.277 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:44:17.278 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-03-21T12:44:17.278 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:44:17.336 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-fuse 2024-03-21T12:44:17.907 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:50 ago on Thu 21 Mar 2024 12:42:27 PM UTC. 2024-03-21T12:44:18.062 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:44:18.063 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:44:18.063 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-03-21T12:44:18.063 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:44:18.063 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-03-21T12:44:18.063 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 85 k 2024-03-21T12:44:18.063 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:18.063 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-03-21T12:44:18.063 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:44:18.063 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-03-21T12:44:18.063 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:18.063 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 85 k 2024-03-21T12:44:18.064 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 231 k 2024-03-21T12:44:18.064 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-03-21T12:44:18.516 INFO:teuthology.orchestra.run.smithi043.stdout:rbd-fuse-19.0.0-2267.g960fbbe4.el9.x86_64.rpm 188 kB/s | 85 kB 00:00 2024-03-21T12:44:18.517 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:44:18.517 INFO:teuthology.orchestra.run.smithi043.stdout:Total 188 kB/s | 85 kB 00:00 2024-03-21T12:44:18.517 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-03-21T12:44:18.525 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-03-21T12:44:18.525 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-03-21T12:44:18.585 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-03-21T12:44:18.585 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-03-21T12:44:18.726 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-03-21T12:44:18.817 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:44:19.194 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:44:19.509 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:44:19.510 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:19.510 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-03-21T12:44:19.510 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:44:19.510 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:19.510 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:44:19.733 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-mirror 2024-03-21T12:44:20.321 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:53 ago on Thu 21 Mar 2024 12:42:27 PM UTC. 2024-03-21T12:44:20.474 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:44:20.475 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:44:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repo Size 2024-03-21T12:44:20.475 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:44:20.475 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-03-21T12:44:20.475 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 3.1 M 2024-03-21T12:44:20.476 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:20.476 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-03-21T12:44:20.476 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:44:20.476 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-03-21T12:44:20.476 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:20.476 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 3.1 M 2024-03-21T12:44:20.476 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 13 M 2024-03-21T12:44:20.476 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-03-21T12:44:21.018 INFO:teuthology.orchestra.run.smithi043.stdout:rbd-mirror-19.0.0-2267.g960fbbe4.el9.x86_64.rpm 5.7 MB/s | 3.1 MB 00:00 2024-03-21T12:44:21.019 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:44:21.020 INFO:teuthology.orchestra.run.smithi043.stdout:Total 5.7 MB/s | 3.1 MB 00:00 2024-03-21T12:44:21.020 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-03-21T12:44:21.029 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-03-21T12:44:21.030 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-03-21T12:44:21.106 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-03-21T12:44:21.106 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-03-21T12:44:21.552 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-03-21T12:44:21.602 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-mirror-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:44:21.627 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:44:21.627 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T12:44:21.627 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-21T12:44:21.627 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T12:44:21.627 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T12:44:21.627 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:22.724 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-mirror-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:44:22.725 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:22.725 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-03-21T12:44:22.725 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:44:22.725 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:22.725 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:44:22.965 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install rbd-nbd 2024-03-21T12:44:23.544 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:01:56 ago on Thu 21 Mar 2024 12:42:27 PM UTC. 2024-03-21T12:44:23.700 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-03-21T12:44:23.700 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:44:23.701 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-03-21T12:44:23.701 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:44:23.701 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-03-21T12:44:23.701 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd x86_64 2:19.0.0-2267.g960fbbe4.el9 ceph 170 k 2024-03-21T12:44:23.701 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:23.701 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-03-21T12:44:23.701 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-03-21T12:44:23.701 INFO:teuthology.orchestra.run.smithi043.stdout:Install 1 Package 2024-03-21T12:44:23.701 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:23.701 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 170 k 2024-03-21T12:44:23.701 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 490 k 2024-03-21T12:44:23.701 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-03-21T12:44:24.089 INFO:teuthology.orchestra.run.smithi043.stdout:rbd-nbd-19.0.0-2267.g960fbbe4.el9.x86_64.rpm 441 kB/s | 170 kB 00:00 2024-03-21T12:44:24.089 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:44:24.089 INFO:teuthology.orchestra.run.smithi043.stdout:Total 439 kB/s | 170 kB 00:00 2024-03-21T12:44:24.089 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-03-21T12:44:24.098 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-03-21T12:44:24.098 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-03-21T12:44:24.156 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-03-21T12:44:24.156 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-03-21T12:44:24.310 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-03-21T12:44:24.398 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-nbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:44:24.892 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:44:25.218 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-nbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 1/1 2024-03-21T12:44:25.218 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:25.218 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-03-21T12:44:25.218 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd-2:19.0.0-2267.g960fbbe4.el9.x86_64 2024-03-21T12:44:25.219 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:44:25.219 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-03-21T12:44:25.453 DEBUG:teuthology.parallel:result is None 2024-03-21T12:44:25.454 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=960fbbe4645c2becc8084669be194c95183c7753 2024-03-21T12:44:25.593 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-21T12:44:25.614 INFO:teuthology.orchestra.run.smithi043.stdout:19.0.0-2267.g960fbbe4.el9 2024-03-21T12:44:25.614 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2267.g960fbbe4.el9 2024-03-21T12:44:25.614 INFO:teuthology.task.install:The correct ceph version 19.0.0-2267.g960fbbe4 is installed. 2024-03-21T12:44:25.615 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=960fbbe4645c2becc8084669be194c95183c7753 2024-03-21T12:44:25.758 DEBUG:teuthology.orchestra.run.smithi155:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-21T12:44:25.783 INFO:teuthology.orchestra.run.smithi155.stdout:19.0.0-2267.g960fbbe4.el9 2024-03-21T12:44:25.783 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2267.g960fbbe4.el9 2024-03-21T12:44:25.783 INFO:teuthology.task.install:The correct ceph version 19.0.0-2267.g960fbbe4 is installed. 2024-03-21T12:44:25.784 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-21T12:44:25.785 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-21T12:44:25.785 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-21T12:44:25.813 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-21T12:44:25.813 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-21T12:44:25.856 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-21T12:44:25.856 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-21T12:44:25.856 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/daemon-helper 2024-03-21T12:44:25.884 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-21T12:44:25.952 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-21T12:44:25.952 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/daemon-helper 2024-03-21T12:44:25.983 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-21T12:44:26.053 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-21T12:44:26.053 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-21T12:44:26.053 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-21T12:44:26.081 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-21T12:44:26.147 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-21T12:44:26.147 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-21T12:44:26.177 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-21T12:44:26.246 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-21T12:44:26.246 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-21T12:44:26.246 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/stdin-killer 2024-03-21T12:44:26.273 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-21T12:44:26.340 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-21T12:44:26.340 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/stdin-killer 2024-03-21T12:44:26.367 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-21T12:44:26.435 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-21T12:44:26.522 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 1, 'ms bind msgr2': False, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'mon_bind_msgr2': False, 'sha1': '960fbbe4645c2becc8084669be194c95183c7753', 'cephadm_mode': 'cephadm-package'} 2024-03-21T12:44:26.523 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 2024-03-21T12:44:26.523 INFO:tasks.cephadm:Cluster fsid is c06c8d1a-e780-11ee-95cd-87774f69a715 2024-03-21T12:44:26.523 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-21T12:44:26.523 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '[v1:172.21.15.43:6789]', 'mon.c': '[v1:172.21.15.43:6790]', 'mon.b': '[v1:172.21.15.155:6789]'} 2024-03-21T12:44:26.523 INFO:tasks.cephadm:First mon is mon.a on smithi043 2024-03-21T12:44:26.523 INFO:tasks.cephadm:First mgr is y 2024-03-21T12:44:26.524 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-21T12:44:26.524 DEBUG:teuthology.orchestra.run.smithi043:> sudo hostname $(hostname -s) 2024-03-21T12:44:26.551 DEBUG:teuthology.orchestra.run.smithi155:> sudo hostname $(hostname -s) 2024-03-21T12:44:26.588 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 on all hosts... 2024-03-21T12:44:26.588 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 pull 2024-03-21T12:44:26.594 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 pull 2024-03-21T12:44:27.414 INFO:teuthology.orchestra.run.smithi155.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753... 2024-03-21T12:44:27.500 INFO:teuthology.orchestra.run.smithi043.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753... 2024-03-21T12:45:35.634 INFO:teuthology.orchestra.run.smithi155.stdout:{ 2024-03-21T12:45:35.635 INFO:teuthology.orchestra.run.smithi155.stdout: "ceph_version": "ceph version 19.0.0-2267-g960fbbe4 (960fbbe4645c2becc8084669be194c95183c7753) squid (dev)", 2024-03-21T12:45:35.635 INFO:teuthology.orchestra.run.smithi155.stdout: "image_id": "19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e", 2024-03-21T12:45:35.635 INFO:teuthology.orchestra.run.smithi155.stdout: "repo_digests": [ 2024-03-21T12:45:35.635 INFO:teuthology.orchestra.run.smithi155.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93" 2024-03-21T12:45:35.635 INFO:teuthology.orchestra.run.smithi155.stdout: ] 2024-03-21T12:45:35.635 INFO:teuthology.orchestra.run.smithi155.stdout:} 2024-03-21T12:45:37.282 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-03-21T12:45:37.282 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-2267-g960fbbe4 (960fbbe4645c2becc8084669be194c95183c7753) squid (dev)", 2024-03-21T12:45:37.282 INFO:teuthology.orchestra.run.smithi043.stdout: "image_id": "19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e", 2024-03-21T12:45:37.282 INFO:teuthology.orchestra.run.smithi043.stdout: "repo_digests": [ 2024-03-21T12:45:37.282 INFO:teuthology.orchestra.run.smithi043.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93" 2024-03-21T12:45:37.282 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-03-21T12:45:37.282 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-03-21T12:45:37.308 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /etc/ceph 2024-03-21T12:45:37.344 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /etc/ceph 2024-03-21T12:45:37.379 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 777 /etc/ceph 2024-03-21T12:45:37.411 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 777 /etc/ceph 2024-03-21T12:45:37.449 INFO:tasks.cephadm:Writing seed config... 2024-03-21T12:45:37.451 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-21T12:45:37.451 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-21T12:45:37.451 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-21T12:45:37.451 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-03-21T12:45:37.451 INFO:tasks.cephadm: override: [global] ms bind msgr2 = False 2024-03-21T12:45:37.452 INFO:tasks.cephadm: override: [global] ms type = async 2024-03-21T12:45:37.452 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-21T12:45:37.452 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-21T12:45:37.452 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-21T12:45:37.452 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-03-21T12:45:37.452 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-21T12:45:37.452 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-21T12:45:37.452 INFO:tasks.cephadm: override: [osd] osd class default list = * 2024-03-21T12:45:37.452 INFO:tasks.cephadm: override: [osd] osd class load list = * 2024-03-21T12:45:37.452 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-21T12:45:37.453 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-21T12:45:37.453 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-21T12:45:37.468 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = c06c8d1a-e780-11ee-95cd-87774f69a715 mon election default strategy = 1 ms bind msgr2 = False ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd class default list = * osd class load list = * [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-03-21T12:45:37.468 DEBUG:teuthology.orchestra.run.smithi043:mon.a> sudo journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mon.a.service 2024-03-21T12:45:37.511 DEBUG:teuthology.orchestra.run.smithi043:mgr.y> sudo journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mgr.y.service 2024-03-21T12:45:37.553 INFO:tasks.cephadm:Bootstrapping... 2024-03-21T12:45:37.553 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 -v bootstrap --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-addrv '[v1:172.21.15.43:6789]' --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-21T12:45:37.748 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-03-21T12:45:37.749 INFO:teuthology.orchestra.run.smithi043.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753', '-v', 'bootstrap', '--fsid', 'c06c8d1a-e780-11ee-95cd-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-addrv', '[v1:172.21.15.43:6789]', '--skip-admin-label'] 2024-03-21T12:45:37.749 INFO:teuthology.orchestra.run.smithi043.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-21T12:45:37.749 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying podman|docker is present... 2024-03-21T12:45:37.774 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-21T12:45:37.774 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying lvm2 is present... 2024-03-21T12:45:37.774 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying time synchronization is in place... 2024-03-21T12:45:37.782 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-21T12:45:37.782 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-21T12:45:37.787 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-21T12:45:37.788 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-03-21T12:45:37.794 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout enabled 2024-03-21T12:45:37.799 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout active 2024-03-21T12:45:37.799 INFO:teuthology.orchestra.run.smithi043.stdout:Unit chronyd.service is enabled and running 2024-03-21T12:45:37.800 INFO:teuthology.orchestra.run.smithi043.stdout:Repeating the final host check... 2024-03-21T12:45:37.823 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-21T12:45:37.823 INFO:teuthology.orchestra.run.smithi043.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-21T12:45:37.823 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl is present 2024-03-21T12:45:37.823 INFO:teuthology.orchestra.run.smithi043.stdout:lvcreate is present 2024-03-21T12:45:37.829 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-21T12:45:37.829 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-21T12:45:37.835 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-21T12:45:37.835 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-03-21T12:45:37.841 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout enabled 2024-03-21T12:45:37.847 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout active 2024-03-21T12:45:37.847 INFO:teuthology.orchestra.run.smithi043.stdout:Unit chronyd.service is enabled and running 2024-03-21T12:45:37.847 INFO:teuthology.orchestra.run.smithi043.stdout:Host looks OK 2024-03-21T12:45:37.847 INFO:teuthology.orchestra.run.smithi043.stdout:Cluster fsid: c06c8d1a-e780-11ee-95cd-87774f69a715 2024-03-21T12:45:37.847 INFO:teuthology.orchestra.run.smithi043.stdout:Acquiring lock 140224586736208 on /run/cephadm/c06c8d1a-e780-11ee-95cd-87774f69a715.lock 2024-03-21T12:45:37.847 INFO:teuthology.orchestra.run.smithi043.stdout:Lock 140224586736208 acquired on /run/cephadm/c06c8d1a-e780-11ee-95cd-87774f69a715.lock 2024-03-21T12:45:37.847 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying IP 172.21.15.43 port 6789 ... 2024-03-21T12:45:37.848 INFO:teuthology.orchestra.run.smithi043.stdout:Base mon IP(s) is [172.21.15.43:6789], mon addrv is [v1:172.21.15.43:6789] 2024-03-21T12:45:37.851 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.43 metric 100 2024-03-21T12:45:37.851 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.43 metric 100 2024-03-21T12:45:37.853 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-21T12:45:37.853 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-03-21T12:45:37.853 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-03-21T12:45:37.856 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-21T12:45:37.856 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-21T12:45:37.856 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-21T12:45:37.856 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-03-21T12:45:37.856 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-03-21T12:45:37.856 INFO:teuthology.orchestra.run.smithi043.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-21T12:45:37.857 INFO:teuthology.orchestra.run.smithi043.stdout:Mon IP `172.21.15.43` is in CIDR network `172.21.0.0/20` 2024-03-21T12:45:37.857 INFO:teuthology.orchestra.run.smithi043.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20'] 2024-03-21T12:45:37.857 INFO:teuthology.orchestra.run.smithi043.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-21T12:45:37.858 INFO:teuthology.orchestra.run.smithi043.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753... 2024-03-21T12:45:38.360 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stdout 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e 2024-03-21T12:45:38.360 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753... 2024-03-21T12:45:38.360 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Getting image source signatures 2024-03-21T12:45:38.360 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying blob sha256:42ed92a376f0fc24f900533b2c051b7c28b467ca85a1aa456c3a11e39d805f31 2024-03-21T12:45:38.360 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-21T12:45:38.360 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Copying config sha256:19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e 2024-03-21T12:45:38.360 INFO:teuthology.orchestra.run.smithi043.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-21T12:45:39.372 INFO:teuthology.orchestra.run.smithi043.stdout:ceph: stdout ceph version 19.0.0-2267-g960fbbe4 (960fbbe4645c2becc8084669be194c95183c7753) squid (dev) 2024-03-21T12:45:39.372 INFO:teuthology.orchestra.run.smithi043.stdout:Ceph version: ceph version 19.0.0-2267-g960fbbe4 (960fbbe4645c2becc8084669be194c95183c7753) squid (dev) 2024-03-21T12:45:39.372 INFO:teuthology.orchestra.run.smithi043.stdout:Extracting ceph user uid/gid from container image... 2024-03-21T12:45:40.360 INFO:teuthology.orchestra.run.smithi043.stdout:stat: stdout 167 167 2024-03-21T12:45:40.361 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial keys... 2024-03-21T12:45:41.326 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQD0K/xlOssBKxAAW35ViAC4lTRMG7lnuXqgtA== 2024-03-21T12:45:42.381 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQD1K/xl6OwJKhAAEWk25YXa0VGFBnu+14Z9YA== 2024-03-21T12:45:43.320 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph-authtool: stdout AQD2K/xlW/rSLBAA1qS5Gyrz4qebJRLygDQ+DA== 2024-03-21T12:45:43.321 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial monmap... 2024-03-21T12:45:44.343 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-21T12:45:44.343 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-21T12:45:44.343 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to c06c8d1a-e780-11ee-95cd-87774f69a715 2024-03-21T12:45:44.343 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-21T12:45:44.343 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool for a [v1:172.21.15.43:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-21T12:45:44.344 INFO:teuthology.orchestra.run.smithi043.stdout:setting min_mon_release = quincy 2024-03-21T12:45:44.344 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: set fsid to c06c8d1a-e780-11ee-95cd-87774f69a715 2024-03-21T12:45:44.344 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-21T12:45:44.344 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:45:44.344 INFO:teuthology.orchestra.run.smithi043.stdout:Creating mon... 2024-03-21T12:45:45.256 INFO:teuthology.orchestra.run.smithi043.stdout:create mon.a on 2024-03-21T12:45:45.455 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-21T12:45:45.597 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-21T12:45:45.758 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-c06c8d1a-e780-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-c06c8d1a-e780-11ee-95cd-87774f69a715.target. 2024-03-21T12:45:45.758 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-c06c8d1a-e780-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-c06c8d1a-e780-11ee-95cd-87774f69a715.target. 2024-03-21T12:45:46.020 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mon.a 2024-03-21T12:45:46.020 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mon.a.service: Unit ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mon.a.service not loaded. 2024-03-21T12:45:46.187 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c06c8d1a-e780-11ee-95cd-87774f69a715.target.wants/ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mon.a.service → /etc/systemd/system/ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@.service. 2024-03-21T12:45:46.726 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T12:45:46.726 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-03-21T12:45:46.732 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T12:45:46.732 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-03-21T12:45:46.733 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-03-21T12:45:46.733 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to enable service . firewalld.service is not available 2024-03-21T12:45:46.733 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mon to start... 2024-03-21T12:45:46.733 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mon... 2024-03-21T12:45:47.782 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout cluster: 2024-03-21T12:45:47.783 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout id: c06c8d1a-e780-11ee-95cd-87774f69a715 2024-03-21T12:45:47.783 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-21T12:45:47.783 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-21T12:45:47.783 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout services: 2024-03-21T12:45:47.783 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.581174s) 2024-03-21T12:45:47.783 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-21T12:45:47.783 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-21T12:45:47.783 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-21T12:45:47.783 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout data: 2024-03-21T12:45:47.783 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-21T12:45:47.784 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-21T12:45:47.784 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-21T12:45:47.784 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout pgs: 2024-03-21T12:45:47.784 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-21T12:45:47.784 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.188+0000 7f1a2faf3700 1 Processor -- start 2024-03-21T12:45:47.784 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.188+0000 7f1a2faf3700 1 -- start start 2024-03-21T12:45:47.784 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.188+0000 7f1a2faf3700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1a2810a410 con 0x7f1a28105fe0 2024-03-21T12:45:47.784 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.188+0000 7f1a2d88f700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1a28105fe0 0x7f1a28106400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43308/0 (socket says 172.21.15.43:43308) 2024-03-21T12:45:47.784 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.188+0000 7f1a2d88f700 1 -- 172.21.15.43:0/1004826322 learned_addr learned my addr 172.21.15.43:0/1004826322 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:45:47.784 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.189+0000 7f1a2c88d700 1 -- 172.21.15.43:0/1004826322 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3413958600 0 0) 0x7f1a2810a410 con 0x7f1a28105fe0 2024-03-21T12:45:47.784 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.189+0000 7f1a2c88d700 1 -- 172.21.15.43:0/1004826322 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a140035b0 con 0x7f1a28105fe0 2024-03-21T12:45:47.785 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.189+0000 7f1a2c88d700 1 -- 172.21.15.43:0/1004826322 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1536400018 0 0) 0x7f1a140035b0 con 0x7f1a28105fe0 2024-03-21T12:45:47.785 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.189+0000 7f1a2c88d700 1 -- 172.21.15.43:0/1004826322 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2810b5f0 con 0x7f1a28105fe0 2024-03-21T12:45:47.785 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.189+0000 7f1a2c88d700 1 -- 172.21.15.43:0/1004826322 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f1a24001580 con 0x7f1a28105fe0 2024-03-21T12:45:47.785 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.189+0000 7f1a2c88d700 1 -- 172.21.15.43:0/1004826322 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1a240018b0 con 0x7f1a28105fe0 2024-03-21T12:45:47.785 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.189+0000 7f1a2c88d700 1 -- 172.21.15.43:0/1004826322 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f1a24001c30 con 0x7f1a28105fe0 2024-03-21T12:45:47.785 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.190+0000 7f1a2faf3700 1 -- 172.21.15.43:0/1004826322 >> v1:172.21.15.43:6789/0 conn(0x7f1a28105fe0 legacy=0x7f1a28106400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:45:47.785 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.190+0000 7f1a2faf3700 1 -- 172.21.15.43:0/1004826322 shutdown_connections 2024-03-21T12:45:47.785 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.190+0000 7f1a2faf3700 1 -- 172.21.15.43:0/1004826322 >> 172.21.15.43:0/1004826322 conn(0x7f1a28101640 msgr2=0x7f1a28103a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:45:47.785 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.190+0000 7f1a2faf3700 1 -- 172.21.15.43:0/1004826322 shutdown_connections 2024-03-21T12:45:47.785 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.190+0000 7f1a2faf3700 1 -- 172.21.15.43:0/1004826322 wait complete. 2024-03-21T12:45:47.785 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.190+0000 7f1a2faf3700 1 Processor -- start 2024-03-21T12:45:47.786 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.191+0000 7f1a2faf3700 1 -- start start 2024-03-21T12:45:47.786 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.191+0000 7f1a2faf3700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1a2819f2e0 con 0x7f1a28105fe0 2024-03-21T12:45:47.786 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.191+0000 7f1a2d88f700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1a28105fe0 0x7f1a2807bc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43324/0 (socket says 172.21.15.43:43324) 2024-03-21T12:45:47.786 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.191+0000 7f1a2d88f700 1 -- 172.21.15.43:0/539508615 learned_addr learned my addr 172.21.15.43:0/539508615 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:45:47.786 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.191+0000 7f1a1effd700 1 -- 172.21.15.43:0/539508615 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2003726428 0 0) 0x7f1a2819f2e0 con 0x7f1a28105fe0 2024-03-21T12:45:47.786 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.191+0000 7f1a1effd700 1 -- 172.21.15.43:0/539508615 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a140031f0 con 0x7f1a28105fe0 2024-03-21T12:45:47.786 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.191+0000 7f1a1effd700 1 -- 172.21.15.43:0/539508615 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2987924689 0 0) 0x7f1a140031f0 con 0x7f1a28105fe0 2024-03-21T12:45:47.786 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.191+0000 7f1a1effd700 1 -- 172.21.15.43:0/539508615 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1a2819f2e0 con 0x7f1a28105fe0 2024-03-21T12:45:47.786 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.191+0000 7f1a1effd700 1 -- 172.21.15.43:0/539508615 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f1a24001480 con 0x7f1a28105fe0 2024-03-21T12:45:47.786 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.191+0000 7f1a1effd700 1 -- 172.21.15.43:0/539508615 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2542918896 0 0) 0x7f1a2819f2e0 con 0x7f1a28105fe0 2024-03-21T12:45:47.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.191+0000 7f1a1effd700 1 -- 172.21.15.43:0/539508615 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a281a04c0 con 0x7f1a28105fe0 2024-03-21T12:45:47.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.191+0000 7f1a2faf3700 1 -- 172.21.15.43:0/539508615 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a2819f4b0 con 0x7f1a28105fe0 2024-03-21T12:45:47.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.192+0000 7f1a2faf3700 1 -- 172.21.15.43:0/539508615 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a2819f9f0 con 0x7f1a28105fe0 2024-03-21T12:45:47.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.192+0000 7f1a1effd700 1 -- 172.21.15.43:0/539508615 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f1a240018b0 con 0x7f1a28105fe0 2024-03-21T12:45:47.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.192+0000 7f1a1effd700 1 -- 172.21.15.43:0/539508615 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f1a24004d90 con 0x7f1a28105fe0 2024-03-21T12:45:47.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.192+0000 7f1a1effd700 1 -- 172.21.15.43:0/539508615 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 1) v1 ==== 807+0+0 (unknown 3189685207 0 0) 0x7f1a24005340 con 0x7f1a28105fe0 2024-03-21T12:45:47.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.192+0000 7f1a1effd700 1 -- 172.21.15.43:0/539508615 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 400852023 0 0) 0x7f1a24006ad0 con 0x7f1a28105fe0 2024-03-21T12:45:47.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.192+0000 7f1a2faf3700 1 -- 172.21.15.43:0/539508615 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a0c005150 con 0x7f1a28105fe0 2024-03-21T12:45:47.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.195+0000 7f1a1effd700 1 -- 172.21.15.43:0/539508615 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7f1a24005810 con 0x7f1a28105fe0 2024-03-21T12:45:47.787 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.269+0000 7f1a2faf3700 1 -- 172.21.15.43:0/539508615 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f1a0c005c50 con 0x7f1a28105fe0 2024-03-21T12:45:47.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.269+0000 7f1a1effd700 1 -- 172.21.15.43:0/539508615 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (unknown 1155462804 0 258055326) 0x7f1a24005a70 con 0x7f1a28105fe0 2024-03-21T12:45:47.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.270+0000 7f1a2faf3700 1 -- 172.21.15.43:0/539508615 >> v1:172.21.15.43:6789/0 conn(0x7f1a28105fe0 legacy=0x7f1a2807bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:45:47.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.270+0000 7f1a2faf3700 1 -- 172.21.15.43:0/539508615 shutdown_connections 2024-03-21T12:45:47.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.270+0000 7f1a2faf3700 1 -- 172.21.15.43:0/539508615 >> 172.21.15.43:0/539508615 conn(0x7f1a28101640 msgr2=0x7f1a28190530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:45:47.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.270+0000 7f1a2faf3700 1 -- 172.21.15.43:0/539508615 shutdown_connections 2024-03-21T12:45:47.788 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:47.270+0000 7f1a2faf3700 1 -- 172.21.15.43:0/539508615 wait complete. 2024-03-21T12:45:47.788 INFO:teuthology.orchestra.run.smithi043.stdout:mon is available 2024-03-21T12:45:47.788 INFO:teuthology.orchestra.run.smithi043.stdout:Assimilating anything we can from ceph.conf... 2024-03-21T12:45:48.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:45:48 smithi043 ceph-mon[29571]: from='client.? v1:172.21.15.43:0/3767577385' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-21T12:45:49.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:45:48 smithi043 ceph-mon[29571]: from='client.? v1:172.21.15.43:0/3767577385' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-21T12:45:51.688 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-21T12:45:51.688 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [global] 2024-03-21T12:45:51.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout fsid = c06c8d1a-e780-11ee-95cd-87774f69a715 2024-03-21T12:45:51.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_host = [v1:172.21.15.43:6789] 2024-03-21T12:45:51.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-21T12:45:51.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-21T12:45:51.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-21T12:45:51.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-21T12:45:51.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-21T12:45:51.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-21T12:45:51.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-21T12:45:51.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-21T12:45:51.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [osd] 2024-03-21T12:45:51.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-21T12:45:51.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-21T12:45:51.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.221+0000 7fdde77a6700 1 Processor -- start 2024-03-21T12:45:51.689 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.222+0000 7fdde77a6700 1 -- start start 2024-03-21T12:45:51.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.222+0000 7fdde77a6700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdde010a450 con 0x7fdde0106020 2024-03-21T12:45:51.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.222+0000 7fdde5542700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdde0106020 0x7fdde0106440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43336/0 (socket says 172.21.15.43:43336) 2024-03-21T12:45:51.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.222+0000 7fdde5542700 1 -- 172.21.15.43:0/1812611825 learned_addr learned my addr 172.21.15.43:0/1812611825 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:45:51.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.222+0000 7fddd3fff700 1 -- 172.21.15.43:0/1812611825 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4250280500 0 0) 0x7fdde010a450 con 0x7fdde0106020 2024-03-21T12:45:51.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.222+0000 7fddd3fff700 1 -- 172.21.15.43:0/1812611825 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddcc0035b0 con 0x7fdde0106020 2024-03-21T12:45:51.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.222+0000 7fddd3fff700 1 -- 172.21.15.43:0/1812611825 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 855766539 0 0) 0x7fddcc0035b0 con 0x7fdde0106020 2024-03-21T12:45:51.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.223+0000 7fddd3fff700 1 -- 172.21.15.43:0/1812611825 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde010b630 con 0x7fdde0106020 2024-03-21T12:45:51.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.223+0000 7fddd3fff700 1 -- 172.21.15.43:0/1812611825 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fdddc001580 con 0x7fdde0106020 2024-03-21T12:45:51.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.223+0000 7fddd3fff700 1 -- 172.21.15.43:0/1812611825 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdddc0017e0 con 0x7fdde0106020 2024-03-21T12:45:51.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.223+0000 7fddd3fff700 1 -- 172.21.15.43:0/1812611825 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fdddc001b60 con 0x7fdde0106020 2024-03-21T12:45:51.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.223+0000 7fdde77a6700 1 -- 172.21.15.43:0/1812611825 >> v1:172.21.15.43:6789/0 conn(0x7fdde0106020 legacy=0x7fdde0106440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:45:51.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.223+0000 7fdde77a6700 1 -- 172.21.15.43:0/1812611825 shutdown_connections 2024-03-21T12:45:51.690 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.223+0000 7fdde77a6700 1 -- 172.21.15.43:0/1812611825 >> 172.21.15.43:0/1812611825 conn(0x7fdde01015e0 msgr2=0x7fdde0103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:45:51.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.223+0000 7fdde77a6700 1 -- 172.21.15.43:0/1812611825 shutdown_connections 2024-03-21T12:45:51.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.223+0000 7fdde77a6700 1 -- 172.21.15.43:0/1812611825 wait complete. 2024-03-21T12:45:51.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.224+0000 7fdde77a6700 1 Processor -- start 2024-03-21T12:45:51.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.224+0000 7fdde77a6700 1 -- start start 2024-03-21T12:45:51.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.224+0000 7fdde77a6700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdde019f300 con 0x7fdde0106020 2024-03-21T12:45:51.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.224+0000 7fdde5542700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdde0106020 0x7fdde007bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43340/0 (socket says 172.21.15.43:43340) 2024-03-21T12:45:51.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.224+0000 7fdde5542700 1 -- 172.21.15.43:0/3767577385 learned_addr learned my addr 172.21.15.43:0/3767577385 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:45:51.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.224+0000 7fddd27fc700 1 -- 172.21.15.43:0/3767577385 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378999483 0 0) 0x7fdde019f300 con 0x7fdde0106020 2024-03-21T12:45:51.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.224+0000 7fddd27fc700 1 -- 172.21.15.43:0/3767577385 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fddcc0031f0 con 0x7fdde0106020 2024-03-21T12:45:51.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.225+0000 7fddd27fc700 1 -- 172.21.15.43:0/3767577385 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4139831588 0 0) 0x7fddcc0031f0 con 0x7fdde0106020 2024-03-21T12:45:51.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.225+0000 7fddd27fc700 1 -- 172.21.15.43:0/3767577385 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdde019f300 con 0x7fdde0106020 2024-03-21T12:45:51.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.225+0000 7fddd27fc700 1 -- 172.21.15.43:0/3767577385 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fdddc001540 con 0x7fdde0106020 2024-03-21T12:45:51.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.225+0000 7fddd27fc700 1 -- 172.21.15.43:0/3767577385 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2645912290 0 0) 0x7fdde019f300 con 0x7fdde0106020 2024-03-21T12:45:51.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.225+0000 7fddd27fc700 1 -- 172.21.15.43:0/3767577385 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdde01a04e0 con 0x7fdde0106020 2024-03-21T12:45:51.691 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.225+0000 7fdde77a6700 1 -- 172.21.15.43:0/3767577385 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdde019f530 con 0x7fdde0106020 2024-03-21T12:45:51.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.225+0000 7fdde77a6700 1 -- 172.21.15.43:0/3767577385 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdde019fa70 con 0x7fdde0106020 2024-03-21T12:45:51.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.225+0000 7fddd27fc700 1 -- 172.21.15.43:0/3767577385 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fdddc0017a0 con 0x7fdde0106020 2024-03-21T12:45:51.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.225+0000 7fddd27fc700 1 -- 172.21.15.43:0/3767577385 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fdddc004c30 con 0x7fdde0106020 2024-03-21T12:45:51.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.226+0000 7fddd27fc700 1 -- 172.21.15.43:0/3767577385 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 1) v1 ==== 807+0+0 (unknown 3189685207 0 0) 0x7fdddc005220 con 0x7fdde0106020 2024-03-21T12:45:51.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.226+0000 7fddd27fc700 1 -- 172.21.15.43:0/3767577385 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 400852023 0 0) 0x7fdddc006930 con 0x7fdde0106020 2024-03-21T12:45:51.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.226+0000 7fdde77a6700 1 -- 172.21.15.43:0/3767577385 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc4005150 con 0x7fdde0106020 2024-03-21T12:45:51.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.228+0000 7fddd27fc700 1 -- 172.21.15.43:0/3767577385 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7fdddc006b30 con 0x7fdde0106020 2024-03-21T12:45:51.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.300+0000 7fdde77a6700 1 -- 172.21.15.43:0/3767577385 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fddc4002430 con 0x7fdde0106020 2024-03-21T12:45:51.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.303+0000 7fddd27fc700 1 -- 172.21.15.43:0/3767577385 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+317 (unknown 1213389831 0 1071483828) 0x7fdddc0058b0 con 0x7fdde0106020 2024-03-21T12:45:51.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.304+0000 7fddd27fc700 1 -- 172.21.15.43:0/3767577385 <== mon.0 v1:172.21.15.43:6789/0 11 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7fdddc005f80 con 0x7fdde0106020 2024-03-21T12:45:51.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.304+0000 7fdde77a6700 1 -- 172.21.15.43:0/3767577385 >> v1:172.21.15.43:6789/0 conn(0x7fdde0106020 legacy=0x7fdde007bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:45:51.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.304+0000 7fdde77a6700 1 -- 172.21.15.43:0/3767577385 shutdown_connections 2024-03-21T12:45:51.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.304+0000 7fdde77a6700 1 -- 172.21.15.43:0/3767577385 >> 172.21.15.43:0/3767577385 conn(0x7fdde01015e0 msgr2=0x7fdde0190570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:45:51.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.305+0000 7fdde77a6700 1 -- 172.21.15.43:0/3767577385 shutdown_connections 2024-03-21T12:45:51.692 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:48.305+0000 7fdde77a6700 1 -- 172.21.15.43:0/3767577385 wait complete. 2024-03-21T12:45:51.693 INFO:teuthology.orchestra.run.smithi043.stdout:Generating new minimal ceph.conf... 2024-03-21T12:45:54.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:45:54 smithi043 ceph-mon[29571]: from='client.? v1:172.21.15.43:0/2560791941' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:46:00.028 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.451+0000 7fcfff9cc700 1 Processor -- start 2024-03-21T12:46:00.028 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.451+0000 7fcfff9cc700 1 -- start start 2024-03-21T12:46:00.028 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.452+0000 7fcfff9cc700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcff8079fa0 con 0x7fcff8106a70 2024-03-21T12:46:00.028 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.452+0000 7fcffd768700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcff8106a70 0x7fcff8108e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60740/0 (socket says 172.21.15.43:60740) 2024-03-21T12:46:00.028 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.452+0000 7fcffd768700 1 -- 172.21.15.43:0/1684980548 learned_addr learned my addr 172.21.15.43:0/1684980548 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:00.028 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.452+0000 7fcfeffff700 1 -- 172.21.15.43:0/1684980548 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 856637714 0 0) 0x7fcff8079fa0 con 0x7fcff8106a70 2024-03-21T12:46:00.028 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.452+0000 7fcfeffff700 1 -- 172.21.15.43:0/1684980548 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfe40035b0 con 0x7fcff8106a70 2024-03-21T12:46:00.028 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.453+0000 7fcfeffff700 1 -- 172.21.15.43:0/1684980548 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1233437660 0 0) 0x7fcfe40035b0 con 0x7fcff8106a70 2024-03-21T12:46:00.029 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.453+0000 7fcfeffff700 1 -- 172.21.15.43:0/1684980548 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff810b590 con 0x7fcff8106a70 2024-03-21T12:46:00.029 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.453+0000 7fcfeffff700 1 -- 172.21.15.43:0/1684980548 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fcff4001580 con 0x7fcff8106a70 2024-03-21T12:46:00.029 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.453+0000 7fcfeffff700 1 -- 172.21.15.43:0/1684980548 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7fcff4001d20 con 0x7fcff8106a70 2024-03-21T12:46:00.029 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.453+0000 7fcfeffff700 1 -- 172.21.15.43:0/1684980548 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fcff4005960 con 0x7fcff8106a70 2024-03-21T12:46:00.029 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.453+0000 7fcfff9cc700 1 -- 172.21.15.43:0/1684980548 >> v1:172.21.15.43:6789/0 conn(0x7fcff8106a70 legacy=0x7fcff8108e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:00.029 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.453+0000 7fcfff9cc700 1 -- 172.21.15.43:0/1684980548 shutdown_connections 2024-03-21T12:46:00.029 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.453+0000 7fcfff9cc700 1 -- 172.21.15.43:0/1684980548 >> 172.21.15.43:0/1684980548 conn(0x7fcff8100440 msgr2=0x7fcff81028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:00.029 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.453+0000 7fcfff9cc700 1 -- 172.21.15.43:0/1684980548 shutdown_connections 2024-03-21T12:46:00.029 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.453+0000 7fcfff9cc700 1 -- 172.21.15.43:0/1684980548 wait complete. 2024-03-21T12:46:00.029 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.454+0000 7fcfff9cc700 1 Processor -- start 2024-03-21T12:46:00.029 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.454+0000 7fcfff9cc700 1 -- start start 2024-03-21T12:46:00.029 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.454+0000 7fcfff9cc700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcff819fad0 con 0x7fcff8106a70 2024-03-21T12:46:00.029 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.454+0000 7fcffd768700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcff8106a70 0x7fcff81062f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60750/0 (socket says 172.21.15.43:60750) 2024-03-21T12:46:00.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.454+0000 7fcffd768700 1 -- 172.21.15.43:0/2560791941 learned_addr learned my addr 172.21.15.43:0/2560791941 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:00.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.454+0000 7fcfee7fc700 1 -- 172.21.15.43:0/2560791941 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 184463845 0 0) 0x7fcff819fad0 con 0x7fcff8106a70 2024-03-21T12:46:00.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.454+0000 7fcfee7fc700 1 -- 172.21.15.43:0/2560791941 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfe40055f0 con 0x7fcff8106a70 2024-03-21T12:46:00.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.455+0000 7fcfee7fc700 1 -- 172.21.15.43:0/2560791941 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1851890364 0 0) 0x7fcfe40055f0 con 0x7fcff8106a70 2024-03-21T12:46:00.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.455+0000 7fcfee7fc700 1 -- 172.21.15.43:0/2560791941 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcff819fad0 con 0x7fcff8106a70 2024-03-21T12:46:00.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.455+0000 7fcfee7fc700 1 -- 172.21.15.43:0/2560791941 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fcff4004870 con 0x7fcff8106a70 2024-03-21T12:46:00.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.455+0000 7fcfee7fc700 1 -- 172.21.15.43:0/2560791941 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1201697403 0 0) 0x7fcff819fad0 con 0x7fcff8106a70 2024-03-21T12:46:00.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.455+0000 7fcfee7fc700 1 -- 172.21.15.43:0/2560791941 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff81a0cb0 con 0x7fcff8106a70 2024-03-21T12:46:00.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.455+0000 7fcfff9cc700 1 -- 172.21.15.43:0/2560791941 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff819fd00 con 0x7fcff8106a70 2024-03-21T12:46:00.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.455+0000 7fcfff9cc700 1 -- 172.21.15.43:0/2560791941 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff81a0240 con 0x7fcff8106a70 2024-03-21T12:46:00.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.455+0000 7fcfee7fc700 1 -- 172.21.15.43:0/2560791941 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7fcff4001190 con 0x7fcff8106a70 2024-03-21T12:46:00.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.455+0000 7fcfee7fc700 1 -- 172.21.15.43:0/2560791941 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fcff4005f70 con 0x7fcff8106a70 2024-03-21T12:46:00.030 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.455+0000 7fcfee7fc700 1 -- 172.21.15.43:0/2560791941 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 1) v1 ==== 807+0+0 (unknown 3189685207 0 0) 0x7fcff4006560 con 0x7fcff8106a70 2024-03-21T12:46:00.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.455+0000 7fcfee7fc700 1 -- 172.21.15.43:0/2560791941 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 400852023 0 0) 0x7fcff4007c70 con 0x7fcff8106a70 2024-03-21T12:46:00.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.456+0000 7fcfff9cc700 1 -- 172.21.15.43:0/2560791941 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfdc005150 con 0x7fcff8106a70 2024-03-21T12:46:00.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.458+0000 7fcfee7fc700 1 -- 172.21.15.43:0/2560791941 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7fcff4001c00 con 0x7fcff8106a70 2024-03-21T12:46:00.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.529+0000 7fcfff9cc700 1 -- 172.21.15.43:0/2560791941 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fcfdc005440 con 0x7fcff8106a70 2024-03-21T12:46:00.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.530+0000 7fcfee7fc700 1 -- 172.21.15.43:0/2560791941 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+147 (unknown 1452402520 0 264009805) 0x7fcff4004ab0 con 0x7fcff8106a70 2024-03-21T12:46:00.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.530+0000 7fcfff9cc700 1 -- 172.21.15.43:0/2560791941 >> v1:172.21.15.43:6789/0 conn(0x7fcff8106a70 legacy=0x7fcff81062f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:00.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.531+0000 7fcfff9cc700 1 -- 172.21.15.43:0/2560791941 shutdown_connections 2024-03-21T12:46:00.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.531+0000 7fcfff9cc700 1 -- 172.21.15.43:0/2560791941 >> 172.21.15.43:0/2560791941 conn(0x7fcff8100440 msgr2=0x7fcff8190800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:00.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.531+0000 7fcfff9cc700 1 -- 172.21.15.43:0/2560791941 shutdown_connections 2024-03-21T12:46:00.031 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:45:54.531+0000 7fcfff9cc700 1 -- 172.21.15.43:0/2560791941 wait complete. 2024-03-21T12:46:00.031 INFO:teuthology.orchestra.run.smithi043.stdout:Restarting the monitor... 2024-03-21T12:46:00.293 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:00 smithi043 systemd[1]: Stopping Ceph mon.a for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T12:46:00.573 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:00 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-a[29548]: 2024-03-21T12:46:00.291+0000 7f1b78141700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T12:46:00.573 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:00 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-a[29548]: 2024-03-21T12:46:00.291+0000 7f1b78141700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-21T12:46:00.573 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:00 smithi043 podman[29903]: 2024-03-21 12:46:00.344422068 +0000 UTC m=+0.151946541 container died 4c8c8f0f5cfb2a2191af335b99f59a88c309502a16c0168885f5fb7af23ad10d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-21T12:46:00.823 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:00 smithi043 podman[29903]: 2024-03-21 12:46:00.664229876 +0000 UTC m=+0.471754349 container cleanup 4c8c8f0f5cfb2a2191af335b99f59a88c309502a16c0168885f5fb7af23ad10d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-03-21T12:46:00.823 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:00 smithi043 bash[29903]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-a 2024-03-21T12:46:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:00 smithi043 podman[29922]: 2024-03-21 12:46:00.823615766 +0000 UTC m=+0.479439683 container remove 4c8c8f0f5cfb2a2191af335b99f59a88c309502a16c0168885f5fb7af23ad10d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-21T12:46:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 systemd[1]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-21T12:46:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 systemd[1]: Stopped Ceph mon.a for c06c8d1a-e780-11ee-95cd-87774f69a715. 2024-03-21T12:46:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 systemd[1]: Starting Ceph mon.a for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T12:46:01.543 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 podman[30024]: 2024-03-21 12:46:01.159611712 +0000 UTC m=+0.017418277 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 2024-03-21T12:46:01.543 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 podman[30024]: 2024-03-21 12:46:01.25778383 +0000 UTC m=+0.115590387 container create c6b9cf84086b2ed440fd90cee2d37a154e05c6c0f7b3fc2f3eaeff8099c00d7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, maintainer=Guillaume Abrioux ) 2024-03-21T12:46:01.543 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 podman[30024]: 2024-03-21 12:46:01.451500115 +0000 UTC m=+0.309306683 container init c6b9cf84086b2ed440fd90cee2d37a154e05c6c0f7b3fc2f3eaeff8099c00d7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-a, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-21T12:46:01.543 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 podman[30024]: 2024-03-21 12:46:01.456687058 +0000 UTC m=+0.314493618 container start c6b9cf84086b2ed440fd90cee2d37a154e05c6c0f7b3fc2f3eaeff8099c00d7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-a, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-21T12:46:01.543 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: set uid:gid to 167:167 (ceph:ceph) 2024-03-21T12:46:01.543 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: ceph version 19.0.0-2267-g960fbbe4 (960fbbe4645c2becc8084669be194c95183c7753) squid (dev), process ceph-mon, pid 7 2024-03-21T12:46:01.544 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: pidfile_write: ignore empty --pid-file 2024-03-21T12:46:01.544 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: load: jerasure load: lrc 2024-03-21T12:46:01.544 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: RocksDB version: 7.9.2 2024-03-21T12:46:01.544 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Git sha 0 2024-03-21T12:46:01.544 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Compile date 2024-03-21 08:53:26 2024-03-21T12:46:01.544 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: DB SUMMARY 2024-03-21T12:46:01.544 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: DB Session ID: AJCSR2L9380NY64UV8KU 2024-03-21T12:46:01.544 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: CURRENT file: CURRENT 2024-03-21T12:46:01.544 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: IDENTITY file: IDENTITY 2024-03-21T12:46:01.544 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-21T12:46:01.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-21T12:46:01.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 85880 ; 2024-03-21T12:46:01.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.error_if_exists: 0 2024-03-21T12:46:01.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.create_if_missing: 0 2024-03-21T12:46:01.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.paranoid_checks: 1 2024-03-21T12:46:01.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-21T12:46:01.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-21T12:46:01.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-21T12:46:01.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.env: 0x5584874b1b20 2024-03-21T12:46:01.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.fs: PosixFileSystem 2024-03-21T12:46:01.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.info_log: 0x558488f53940 2024-03-21T12:46:01.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_file_opening_threads: 16 2024-03-21T12:46:01.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.statistics: (nil) 2024-03-21T12:46:01.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.use_fsync: 0 2024-03-21T12:46:01.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_log_file_size: 0 2024-03-21T12:46:01.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-21T12:46:01.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-21T12:46:01.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.keep_log_file_num: 1000 2024-03-21T12:46:01.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.recycle_log_file_num: 0 2024-03-21T12:46:01.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.allow_fallocate: 1 2024-03-21T12:46:01.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.allow_mmap_reads: 0 2024-03-21T12:46:01.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.allow_mmap_writes: 0 2024-03-21T12:46:01.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.use_direct_reads: 0 2024-03-21T12:46:01.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-21T12:46:01.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.create_missing_column_families: 0 2024-03-21T12:46:01.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.db_log_dir: 2024-03-21T12:46:01.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.wal_dir: 2024-03-21T12:46:01.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-21T12:46:01.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-21T12:46:01.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-21T12:46:01.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-21T12:46:01.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-21T12:46:01.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-21T12:46:01.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.advise_random_on_open: 1 2024-03-21T12:46:01.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.db_write_buffer_size: 0 2024-03-21T12:46:01.548 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.write_buffer_manager: 0x558488f241e0 2024-03-21T12:46:01.548 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-21T12:46:01.548 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-21T12:46:01.548 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-21T12:46:01.548 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.rate_limiter: (nil) 2024-03-21T12:46:01.548 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-21T12:46:01.550 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.wal_recovery_mode: 2 2024-03-21T12:46:01.550 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.enable_thread_tracking: 0 2024-03-21T12:46:01.550 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.enable_pipelined_write: 0 2024-03-21T12:46:01.550 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.unordered_write: 0 2024-03-21T12:46:01.550 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-21T12:46:01.550 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-21T12:46:01.550 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-21T12:46:01.551 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-21T12:46:01.551 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.row_cache: None 2024-03-21T12:46:01.551 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.wal_filter: None 2024-03-21T12:46:01.551 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-21T12:46:01.551 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.allow_ingest_behind: 0 2024-03-21T12:46:01.551 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.two_write_queues: 0 2024-03-21T12:46:01.551 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.manual_wal_flush: 0 2024-03-21T12:46:01.551 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.wal_compression: 0 2024-03-21T12:46:01.552 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.atomic_flush: 0 2024-03-21T12:46:01.552 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-21T12:46:01.552 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-21T12:46:01.552 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-21T12:46:01.552 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.log_readahead_size: 0 2024-03-21T12:46:01.552 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-21T12:46:01.553 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.best_efforts_recovery: 0 2024-03-21T12:46:01.553 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-21T12:46:01.553 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-21T12:46:01.553 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.allow_data_in_errors: 0 2024-03-21T12:46:01.553 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.db_host_id: __hostname__ 2024-03-21T12:46:01.553 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-21T12:46:01.553 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_background_jobs: 2 2024-03-21T12:46:01.553 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_background_compactions: -1 2024-03-21T12:46:01.553 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_subcompactions: 1 2024-03-21T12:46:01.554 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-21T12:46:01.554 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-21T12:46:01.554 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-21T12:46:01.554 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_total_wal_size: 0 2024-03-21T12:46:01.554 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-21T12:46:01.554 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-21T12:46:01.554 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-21T12:46:01.554 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-21T12:46:01.554 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_open_files: -1 2024-03-21T12:46:01.554 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.bytes_per_sync: 0 2024-03-21T12:46:01.554 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-21T12:46:01.555 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-21T12:46:01.555 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.compaction_readahead_size: 0 2024-03-21T12:46:01.555 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_background_flushes: -1 2024-03-21T12:46:01.555 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Compression algorithms supported: 2024-03-21T12:46:01.555 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-21T12:46:01.555 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: kZSTD supported: 0 2024-03-21T12:46:01.555 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: kXpressCompression supported: 0 2024-03-21T12:46:01.555 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: kLZ4HCCompression supported: 1 2024-03-21T12:46:01.555 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: kZlibCompression supported: 1 2024-03-21T12:46:01.555 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: kSnappyCompression supported: 1 2024-03-21T12:46:01.555 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: kLZ4Compression supported: 1 2024-03-21T12:46:01.555 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: kBZip2Compression supported: 0 2024-03-21T12:46:01.555 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-21T12:46:01.556 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-21T12:46:01.556 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-21T12:46:01.556 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-21T12:46:01.556 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-21T12:46:01.556 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.merge_operator: 2024-03-21T12:46:01.556 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.compaction_filter: None 2024-03-21T12:46:01.556 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.compaction_filter_factory: None 2024-03-21T12:46:01.556 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.sst_partitioner_factory: None 2024-03-21T12:46:01.556 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-21T12:46:01.556 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-21T12:46:01.556 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558488f53a40) 2024-03-21T12:46:01.557 INFO:journalctl@ceph.mon.a.smithi043.stdout: cache_index_and_filter_blocks: 1 2024-03-21T12:46:01.557 INFO:journalctl@ceph.mon.a.smithi043.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-21T12:46:01.557 INFO:journalctl@ceph.mon.a.smithi043.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-21T12:46:01.557 INFO:journalctl@ceph.mon.a.smithi043.stdout: pin_top_level_index_and_filter: 1 2024-03-21T12:46:01.557 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_type: 0 2024-03-21T12:46:01.557 INFO:journalctl@ceph.mon.a.smithi043.stdout: data_block_index_type: 0 2024-03-21T12:46:01.557 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_shortening: 1 2024-03-21T12:46:01.557 INFO:journalctl@ceph.mon.a.smithi043.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-21T12:46:01.557 INFO:journalctl@ceph.mon.a.smithi043.stdout: checksum: 4 2024-03-21T12:46:01.557 INFO:journalctl@ceph.mon.a.smithi043.stdout: no_block_cache: 0 2024-03-21T12:46:01.557 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache: 0x558488f57090 2024-03-21T12:46:01.557 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_name: BinnedLRUCache 2024-03-21T12:46:01.557 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_options: 2024-03-21T12:46:01.557 INFO:journalctl@ceph.mon.a.smithi043.stdout: capacity : 536870912 2024-03-21T12:46:01.558 INFO:journalctl@ceph.mon.a.smithi043.stdout: num_shard_bits : 4 2024-03-21T12:46:01.558 INFO:journalctl@ceph.mon.a.smithi043.stdout: strict_capacity_limit : 0 2024-03-21T12:46:01.558 INFO:journalctl@ceph.mon.a.smithi043.stdout: high_pri_pool_ratio: 0.000 2024-03-21T12:46:01.558 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_cache_compressed: (nil) 2024-03-21T12:46:01.558 INFO:journalctl@ceph.mon.a.smithi043.stdout: persistent_cache: (nil) 2024-03-21T12:46:01.558 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_size: 4096 2024-03-21T12:46:01.558 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_size_deviation: 10 2024-03-21T12:46:01.558 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_restart_interval: 16 2024-03-21T12:46:01.558 INFO:journalctl@ceph.mon.a.smithi043.stdout: index_block_restart_interval: 1 2024-03-21T12:46:01.558 INFO:journalctl@ceph.mon.a.smithi043.stdout: metadata_block_size: 4096 2024-03-21T12:46:01.558 INFO:journalctl@ceph.mon.a.smithi043.stdout: partition_filters: 0 2024-03-21T12:46:01.558 INFO:journalctl@ceph.mon.a.smithi043.stdout: use_delta_encoding: 1 2024-03-21T12:46:01.558 INFO:journalctl@ceph.mon.a.smithi043.stdout: filter_policy: bloomfilter 2024-03-21T12:46:01.559 INFO:journalctl@ceph.mon.a.smithi043.stdout: whole_key_filtering: 1 2024-03-21T12:46:01.559 INFO:journalctl@ceph.mon.a.smithi043.stdout: verify_compression: 0 2024-03-21T12:46:01.559 INFO:journalctl@ceph.mon.a.smithi043.stdout: read_amp_bytes_per_bit: 0 2024-03-21T12:46:01.559 INFO:journalctl@ceph.mon.a.smithi043.stdout: format_version: 5 2024-03-21T12:46:01.559 INFO:journalctl@ceph.mon.a.smithi043.stdout: enable_index_compression: 1 2024-03-21T12:46:01.559 INFO:journalctl@ceph.mon.a.smithi043.stdout: block_align: 0 2024-03-21T12:46:01.559 INFO:journalctl@ceph.mon.a.smithi043.stdout: max_auto_readahead_size: 262144 2024-03-21T12:46:01.559 INFO:journalctl@ceph.mon.a.smithi043.stdout: prepopulate_block_cache: 0 2024-03-21T12:46:01.559 INFO:journalctl@ceph.mon.a.smithi043.stdout: initial_auto_readahead_size: 8192 2024-03-21T12:46:01.559 INFO:journalctl@ceph.mon.a.smithi043.stdout: num_file_reads_for_auto_readahead: 2 2024-03-21T12:46:01.559 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.write_buffer_size: 33554432 2024-03-21T12:46:01.559 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_write_buffer_number: 2 2024-03-21T12:46:01.559 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.compression: NoCompression 2024-03-21T12:46:01.560 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.bottommost_compression: Disabled 2024-03-21T12:46:01.560 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.prefix_extractor: nullptr 2024-03-21T12:46:01.560 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-21T12:46:01.560 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.num_levels: 7 2024-03-21T12:46:01.560 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-21T12:46:01.560 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-21T12:46:01.560 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-21T12:46:01.560 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-21T12:46:01.560 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-21T12:46:01.560 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-21T12:46:01.560 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-21T12:46:01.560 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-21T12:46:01.560 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-21T12:46:01.560 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-21T12:46:01.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-21T12:46:01.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-21T12:46:01.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-21T12:46:01.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.compression_opts.level: 32767 2024-03-21T12:46:01.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.compression_opts.strategy: 0 2024-03-21T12:46:01.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-21T12:46:01.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-21T12:46:01.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-21T12:46:01.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-21T12:46:01.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.compression_opts.enabled: false 2024-03-21T12:46:01.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-21T12:46:01.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-21T12:46:01.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-21T12:46:01.561 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-21T12:46:01.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.target_file_size_base: 67108864 2024-03-21T12:46:01.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-21T12:46:01.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-21T12:46:01.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-21T12:46:01.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-21T12:46:01.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-21T12:46:01.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-21T12:46:01.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-21T12:46:01.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-21T12:46:01.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-21T12:46:01.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-21T12:46:01.562 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-21T12:46:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-21T12:46:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-21T12:46:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-21T12:46:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.arena_block_size: 1048576 2024-03-21T12:46:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-21T12:46:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-21T12:46:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.disable_auto_compactions: 0 2024-03-21T12:46:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-21T12:46:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-21T12:46:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-21T12:46:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-21T12:46:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-21T12:46:01.563 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-21T12:46:01.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-21T12:46:01.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-21T12:46:01.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-21T12:46:01.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-21T12:46:01.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.table_properties_collectors: 2024-03-21T12:46:01.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.inplace_update_support: 0 2024-03-21T12:46:01.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-21T12:46:01.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-21T12:46:01.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-21T12:46:01.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-21T12:46:01.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.bloom_locality: 0 2024-03-21T12:46:01.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.max_successive_merges: 0 2024-03-21T12:46:01.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-21T12:46:01.564 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.paranoid_file_checks: 0 2024-03-21T12:46:01.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.force_consistency_checks: 1 2024-03-21T12:46:01.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.report_bg_io_stats: 0 2024-03-21T12:46:01.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.ttl: 2592000 2024-03-21T12:46:01.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-21T12:46:01.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-21T12:46:01.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-21T12:46:01.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.enable_blob_files: false 2024-03-21T12:46:01.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.min_blob_size: 0 2024-03-21T12:46:01.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.blob_file_size: 268435456 2024-03-21T12:46:01.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-21T12:46:01.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-21T12:46:01.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-21T12:46:01.565 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-21T12:46:01.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-21T12:46:01.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.blob_file_starting_level: 0 2024-03-21T12:46:01.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-21T12:46:01.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-21T12:46:01.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-21T12:46:01.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 22ce3cdc-52d6-4118-87e7-90ac5138aec4 2024-03-21T12:46:01.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711025161485662, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-21T12:46:01.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-21T12:46:01.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711025161486504, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 82214, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 264, "table_properties": {"data_size": 80413, "index_size": 211, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10703, "raw_average_key_size": 47, "raw_value_size": 74388, "raw_average_value_size": 333, "num_data_blocks": 10, "num_entries": 223, "num_filter_entries": 223, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711025161, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "22ce3cdc-52d6-4118-87e7-90ac5138aec4", "db_session_id": "AJCSR2L9380NY64UV8KU", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-21T12:46:01.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711025161486588, "job": 1, "event": "recovery_finished"} 2024-03-21T12:46:01.566 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-21T12:46:01.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-21T12:46:01.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558489046000 2024-03-21T12:46:01.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: DB pointer 0x558489030000 2024-03-21T12:46:01.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-21T12:46:01.567 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-21T12:46:01.567 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** DB Stats ** 2024-03-21T12:46:01.567 INFO:journalctl@ceph.mon.a.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T12:46:01.567 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-21T12:46:01.567 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T12:46:01.567 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T12:46:01.567 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-21T12:46:01.567 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T12:46:01.567 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T12:46:01.568 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-03-21T12:46:01.568 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** Compaction Stats [default] ** 2024-03-21T12:46:01.568 INFO:journalctl@ceph.mon.a.smithi043.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T12:46:01.568 INFO:journalctl@ceph.mon.a.smithi043.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-21T12:46:01.568 INFO:journalctl@ceph.mon.a.smithi043.stdout: L0 2/0 82.09 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T12:46:01.568 INFO:journalctl@ceph.mon.a.smithi043.stdout: Sum 2/0 82.09 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T12:46:01.568 INFO:journalctl@ceph.mon.a.smithi043.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T12:46:01.568 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-03-21T12:46:01.568 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** Compaction Stats [default] ** 2024-03-21T12:46:01.569 INFO:journalctl@ceph.mon.a.smithi043.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T12:46:01.569 INFO:journalctl@ceph.mon.a.smithi043.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-21T12:46:01.569 INFO:journalctl@ceph.mon.a.smithi043.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 144.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T12:46:01.569 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-03-21T12:46:01.569 INFO:journalctl@ceph.mon.a.smithi043.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-21T12:46:01.569 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-03-21T12:46:01.569 INFO:journalctl@ceph.mon.a.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T12:46:01.569 INFO:journalctl@ceph.mon.a.smithi043.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-21T12:46:01.569 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-21T12:46:01.569 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-21T12:46:01.569 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-21T12:46:01.570 INFO:journalctl@ceph.mon.a.smithi043.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-21T12:46:01.570 INFO:journalctl@ceph.mon.a.smithi043.stdout: Cumulative compaction: 0.00 GB write, 27.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T12:46:01.570 INFO:journalctl@ceph.mon.a.smithi043.stdout: Interval compaction: 0.00 GB write, 27.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T12:46:01.570 INFO:journalctl@ceph.mon.a.smithi043.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-21T12:46:01.570 INFO:journalctl@ceph.mon.a.smithi043.stdout: Block cache BinnedLRUCache@0x558488f57090#7 capacity: 512.00 MB usage: 1.17 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-03-21T12:46:01.570 INFO:journalctl@ceph.mon.a.smithi043.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.41 KB,7.7486e-05%) Misc(1,0.00 KB,0%) 2024-03-21T12:46:01.570 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-03-21T12:46:01.570 INFO:journalctl@ceph.mon.a.smithi043.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-21T12:46:01.570 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: starting mon.a rank 0 at public addrs v1:172.21.15.43:6789/0 at bind addrs v1:172.21.15.43:6789/0 mon_data /var/lib/ceph/mon/ceph-a fsid c06c8d1a-e780-11ee-95cd-87774f69a715 2024-03-21T12:46:01.570 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: mon.a@-1(???) e1 preinit fsid c06c8d1a-e780-11ee-95cd-87774f69a715 2024-03-21T12:46:01.570 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: mon.a@-1(???).mds e1 new map 2024-03-21T12:46:01.571 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: mon.a@-1(???).mds e1 print_map 2024-03-21T12:46:01.571 INFO:journalctl@ceph.mon.a.smithi043.stdout: e1 2024-03-21T12:46:01.571 INFO:journalctl@ceph.mon.a.smithi043.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T12:46:01.571 INFO:journalctl@ceph.mon.a.smithi043.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T12:46:01.571 INFO:journalctl@ceph.mon.a.smithi043.stdout: legacy client fscid: -1 2024-03-21T12:46:01.571 INFO:journalctl@ceph.mon.a.smithi043.stdout: 2024-03-21T12:46:01.571 INFO:journalctl@ceph.mon.a.smithi043.stdout: No filesystems configured 2024-03-21T12:46:01.572 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-21T12:46:01.572 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T12:46:01.572 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T12:46:01.572 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T12:46:01.572 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-21T12:46:01.572 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 bash[30024]: c6b9cf84086b2ed440fd90cee2d37a154e05c6c0f7b3fc2f3eaeff8099c00d7c 2024-03-21T12:46:01.572 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 systemd[1]: Started Ceph mon.a for c06c8d1a-e780-11ee-95cd-87774f69a715. 2024-03-21T12:46:01.572 INFO:teuthology.orchestra.run.smithi043.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-21T12:46:01.803 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-21T12:46:01.803 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: monmap e1: 1 mons at {a=v1:172.21.15.43:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-03-21T12:46:01.803 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: fsmap 2024-03-21T12:46:01.803 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: osdmap e1: 0 total, 0 up, 0 in 2024-03-21T12:46:01.803 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:01 smithi043 ceph-mon[30062]: mgrmap e1: no daemons active 2024-03-21T12:46:02.699 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.029+0000 7f3070be1700 1 Processor -- start 2024-03-21T12:46:02.699 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.029+0000 7f3070be1700 1 -- start start 2024-03-21T12:46:02.699 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.029+0000 7f3070be1700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f306c07abd0 con 0x7f306c07bfd0 2024-03-21T12:46:02.699 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.030+0000 7f306a59c700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f306c07bfd0 0x7f306c07a430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41720/0 (socket says 172.21.15.43:41720) 2024-03-21T12:46:02.699 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.030+0000 7f306a59c700 1 -- 172.21.15.43:0/3063068698 learned_addr learned my addr 172.21.15.43:0/3063068698 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:02.699 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.030+0000 7f306959a700 1 -- 172.21.15.43:0/3063068698 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4021243735 0 0) 0x7f306c07abd0 con 0x7f306c07bfd0 2024-03-21T12:46:02.699 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.030+0000 7f306959a700 1 -- 172.21.15.43:0/3063068698 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30580035b0 con 0x7f306c07bfd0 2024-03-21T12:46:02.699 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.030+0000 7f306959a700 1 -- 172.21.15.43:0/3063068698 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3472907530 0 0) 0x7f30580035b0 con 0x7f306c07bfd0 2024-03-21T12:46:02.699 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.030+0000 7f306959a700 1 -- 172.21.15.43:0/3063068698 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306c07ada0 con 0x7f306c07bfd0 2024-03-21T12:46:02.699 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.030+0000 7f306959a700 1 -- 172.21.15.43:0/3063068698 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f3064001900 con 0x7f306c07bfd0 2024-03-21T12:46:02.699 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.030+0000 7f306959a700 1 -- 172.21.15.43:0/3063068698 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7f30640020a0 con 0x7f306c07bfd0 2024-03-21T12:46:02.699 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.030+0000 7f306959a700 1 -- 172.21.15.43:0/3063068698 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f3064005ce0 con 0x7f306c07bfd0 2024-03-21T12:46:02.699 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.031+0000 7f3070be1700 1 -- 172.21.15.43:0/3063068698 >> v1:172.21.15.43:6789/0 conn(0x7f306c07bfd0 legacy=0x7f306c07a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:02.700 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.031+0000 7f3070be1700 1 -- 172.21.15.43:0/3063068698 shutdown_connections 2024-03-21T12:46:02.700 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.031+0000 7f3070be1700 1 -- 172.21.15.43:0/3063068698 >> 172.21.15.43:0/3063068698 conn(0x7f306c101650 msgr2=0x7f306c103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:02.700 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.031+0000 7f3070be1700 1 -- 172.21.15.43:0/3063068698 shutdown_connections 2024-03-21T12:46:02.700 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.031+0000 7f3070be1700 1 -- 172.21.15.43:0/3063068698 wait complete. 2024-03-21T12:46:02.700 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.032+0000 7f3070be1700 1 Processor -- start 2024-03-21T12:46:02.700 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.032+0000 7f3070be1700 1 -- start start 2024-03-21T12:46:02.700 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.032+0000 7f3070be1700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f306c19c460 con 0x7f306c07bfd0 2024-03-21T12:46:02.700 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.032+0000 7f306a59c700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f306c07bfd0 0x7f306c19bd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41724/0 (socket says 172.21.15.43:41724) 2024-03-21T12:46:02.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.032+0000 7f306a59c700 1 -- 172.21.15.43:0/2509492871 learned_addr learned my addr 172.21.15.43:0/2509492871 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:02.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.032+0000 7f30637fe700 1 -- 172.21.15.43:0/2509492871 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 307951507 0 0) 0x7f306c19c460 con 0x7f306c07bfd0 2024-03-21T12:46:02.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.032+0000 7f30637fe700 1 -- 172.21.15.43:0/2509492871 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30580055f0 con 0x7f306c07bfd0 2024-03-21T12:46:02.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.032+0000 7f30637fe700 1 -- 172.21.15.43:0/2509492871 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1762889360 0 0) 0x7f30580055f0 con 0x7f306c07bfd0 2024-03-21T12:46:02.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.032+0000 7f30637fe700 1 -- 172.21.15.43:0/2509492871 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f306c19c460 con 0x7f306c07bfd0 2024-03-21T12:46:02.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.033+0000 7f30637fe700 1 -- 172.21.15.43:0/2509492871 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f3064001400 con 0x7f306c07bfd0 2024-03-21T12:46:02.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.033+0000 7f30637fe700 1 -- 172.21.15.43:0/2509492871 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2592414403 0 0) 0x7f306c19c460 con 0x7f306c07bfd0 2024-03-21T12:46:02.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.033+0000 7f30637fe700 1 -- 172.21.15.43:0/2509492871 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306c19c630 con 0x7f306c07bfd0 2024-03-21T12:46:02.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.033+0000 7f3070be1700 1 -- 172.21.15.43:0/2509492871 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f306c19c920 con 0x7f306c07bfd0 2024-03-21T12:46:02.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.033+0000 7f3070be1700 1 -- 172.21.15.43:0/2509492871 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f306c1a0410 con 0x7f306c07bfd0 2024-03-21T12:46:02.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.033+0000 7f30637fe700 1 -- 172.21.15.43:0/2509492871 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7f3064001f20 con 0x7f306c07bfd0 2024-03-21T12:46:02.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.033+0000 7f30637fe700 1 -- 172.21.15.43:0/2509492871 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f3064006230 con 0x7f306c07bfd0 2024-03-21T12:46:02.701 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.033+0000 7f30637fe700 1 -- 172.21.15.43:0/2509492871 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 1) v1 ==== 807+0+0 (unknown 3189685207 0 0) 0x7f3064006860 con 0x7f306c07bfd0 2024-03-21T12:46:02.702 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.033+0000 7f30637fe700 1 -- 172.21.15.43:0/2509492871 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 400852023 0 0) 0x7f3064007f40 con 0x7f306c07bfd0 2024-03-21T12:46:02.702 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.034+0000 7f3070be1700 1 -- 172.21.15.43:0/2509492871 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f304c005150 con 0x7f306c07bfd0 2024-03-21T12:46:02.702 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.036+0000 7f30637fe700 1 -- 172.21.15.43:0/2509492871 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7f3064004e30 con 0x7f306c07bfd0 2024-03-21T12:46:02.702 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.106+0000 7f3070be1700 1 -- 172.21.15.43:0/2509492871 --> v1:172.21.15.43:6789/0 -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f304c005440 con 0x7f306c07bfd0 2024-03-21T12:46:02.702 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.108+0000 7f30637fe700 1 -- 172.21.15.43:0/2509492871 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (unknown 534127980 0 0) 0x7f3064005a50 con 0x7f306c07bfd0 2024-03-21T12:46:02.702 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.108+0000 7f3070be1700 1 -- 172.21.15.43:0/2509492871 >> v1:172.21.15.43:6789/0 conn(0x7f306c07bfd0 legacy=0x7f306c19bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:02.702 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.108+0000 7f3070be1700 1 -- 172.21.15.43:0/2509492871 shutdown_connections 2024-03-21T12:46:02.702 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.108+0000 7f3070be1700 1 -- 172.21.15.43:0/2509492871 >> 172.21.15.43:0/2509492871 conn(0x7f306c101650 msgr2=0x7f306c10a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:02.703 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.108+0000 7f3070be1700 1 -- 172.21.15.43:0/2509492871 shutdown_connections 2024-03-21T12:46:02.703 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:02.109+0000 7f3070be1700 1 -- 172.21.15.43:0/2509492871 wait complete. 2024-03-21T12:46:02.703 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-21T12:46:02.723 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-21T12:46:02.724 INFO:teuthology.orchestra.run.smithi043.stdout:Creating mgr... 2024-03-21T12:46:02.724 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-21T12:46:02.724 INFO:teuthology.orchestra.run.smithi043.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-21T12:46:02.978 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mgr.y 2024-03-21T12:46:02.978 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Failed to reset failed state of unit ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mgr.y.service: Unit ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mgr.y.service not loaded. 2024-03-21T12:46:03.123 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-c06c8d1a-e780-11ee-95cd-87774f69a715.target.wants/ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mgr.y.service → /etc/systemd/system/ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@.service. 2024-03-21T12:46:03.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:03 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2509492871' entity='client.admin' 2024-03-21T12:46:03.364 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:03 smithi043 podman[30308]: 2024-03-21 12:46:03.266260394 +0000 UTC m=+0.017453018 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 2024-03-21T12:46:03.628 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:03 smithi043 podman[30308]: 2024-03-21 12:46:03.562753791 +0000 UTC m=+0.313946419 container init a65d78811d7799862d0e8e9234e6ad3abe4cd40b6af9152fb7d496f3b4461778 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-21T12:46:03.629 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:03 smithi043 podman[30308]: 2024-03-21 12:46:03.567762734 +0000 UTC m=+0.318955353 container start a65d78811d7799862d0e8e9234e6ad3abe4cd40b6af9152fb7d496f3b4461778 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.schema-version=1.0) 2024-03-21T12:46:03.642 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T12:46:03.642 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-03-21T12:46:03.649 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T12:46:03.649 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-03-21T12:46:03.649 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-03-21T12:46:03.649 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to enable service . firewalld.service is not available 2024-03-21T12:46:03.658 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T12:46:03.658 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-03-21T12:46:03.666 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T12:46:03.666 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-03-21T12:46:03.666 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-03-21T12:46:03.666 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-21T12:46:03.666 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr to start... 2024-03-21T12:46:03.667 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr... 2024-03-21T12:46:03.889 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:03 smithi043 bash[30308]: a65d78811d7799862d0e8e9234e6ad3abe4cd40b6af9152fb7d496f3b4461778 2024-03-21T12:46:03.889 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:03 smithi043 systemd[1]: Started Ceph mgr.y for c06c8d1a-e780-11ee-95cd-87774f69a715. 2024-03-21T12:46:04.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1696566992' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T12:46:04.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:04 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:04.143+0000 7f35ae99a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T12:46:04.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:04 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:04.242+0000 7f35ae99a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T12:46:04.832 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-21T12:46:04.832 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-03-21T12:46:04.832 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "c06c8d1a-e780-11ee-95cd-87774f69a715", 2024-03-21T12:46:04.832 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T12:46:04.832 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T12:46:04.832 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T12:46:04.833 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T12:46:04.833 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:04.833 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T12:46:04.833 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T12:46:04.833 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-03-21T12:46:04.833 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-03-21T12:46:04.833 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T12:46:04.833 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T12:46:04.833 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-03-21T12:46:04.833 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-21T12:46:04.833 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T12:46:04.833 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T12:46:04.834 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T12:46:04.834 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T12:46:04.834 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:04.834 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T12:46:04.834 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T12:46:04.834 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T12:46:04.834 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T12:46:04.834 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T12:46:04.842 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T12:46:04.842 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T12:46:04.842 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T12:46:04.842 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:04.842 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T12:46:04.842 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T12:46:04.842 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T12:46:04.842 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T12:46:04.843 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T12:46:04.843 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T12:46:04.843 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T12:46:04.843 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T12:46:04.843 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T12:46:04.843 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:04.843 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T12:46:04.843 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T12:46:04.843 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T12:46:04.843 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T12:46:04.843 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:04.843 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T12:46:04.844 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-21T12:46:04.844 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T12:46:04.844 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T12:46:04.844 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T12:46:04.844 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T12:46:04.844 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T12:46:04.844 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-03-21T12:46:04.844 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T12:46:04.844 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:04.844 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T12:46:04.844 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T12:46:04.844 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T12:45:46.690987+0000", 2024-03-21T12:46:04.844 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T12:46:04.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:04.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T12:46:04.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-03-21T12:46:04.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.135+0000 7fb67c985700 1 Processor -- start 2024-03-21T12:46:04.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.136+0000 7fb67c985700 1 -- start start 2024-03-21T12:46:04.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.136+0000 7fb67c985700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb678079f60 con 0x7fb6781048b0 2024-03-21T12:46:04.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.136+0000 7fb67659c700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb6781048b0 0x7fb678104cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41760/0 (socket says 172.21.15.43:41760) 2024-03-21T12:46:04.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.136+0000 7fb67659c700 1 -- 172.21.15.43:0/552547722 learned_addr learned my addr 172.21.15.43:0/552547722 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:04.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.136+0000 7fb67559a700 1 -- 172.21.15.43:0/552547722 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3880284444 0 0) 0x7fb678079f60 con 0x7fb6781048b0 2024-03-21T12:46:04.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.136+0000 7fb67559a700 1 -- 172.21.15.43:0/552547722 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6640035b0 con 0x7fb6781048b0 2024-03-21T12:46:04.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.137+0000 7fb67559a700 1 -- 172.21.15.43:0/552547722 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2642472455 0 0) 0x7fb6640035b0 con 0x7fb6781048b0 2024-03-21T12:46:04.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.137+0000 7fb67559a700 1 -- 172.21.15.43:0/552547722 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6781063f0 con 0x7fb6781048b0 2024-03-21T12:46:04.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.137+0000 7fb67559a700 1 -- 172.21.15.43:0/552547722 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fb668001580 con 0x7fb6781048b0 2024-03-21T12:46:04.845 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.137+0000 7fb67559a700 1 -- 172.21.15.43:0/552547722 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7fb668001d20 con 0x7fb6781048b0 2024-03-21T12:46:04.846 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.137+0000 7fb67559a700 1 -- 172.21.15.43:0/552547722 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fb668005960 con 0x7fb6781048b0 2024-03-21T12:46:04.846 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.137+0000 7fb67c985700 1 -- 172.21.15.43:0/552547722 >> v1:172.21.15.43:6789/0 conn(0x7fb6781048b0 legacy=0x7fb678104cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:04.846 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.137+0000 7fb67c985700 1 -- 172.21.15.43:0/552547722 shutdown_connections 2024-03-21T12:46:04.846 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.137+0000 7fb67c985700 1 -- 172.21.15.43:0/552547722 >> 172.21.15.43:0/552547722 conn(0x7fb678100470 msgr2=0x7fb6781028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:04.846 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.138+0000 7fb67c985700 1 -- 172.21.15.43:0/552547722 shutdown_connections 2024-03-21T12:46:04.846 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.138+0000 7fb67c985700 1 -- 172.21.15.43:0/552547722 wait complete. 2024-03-21T12:46:04.846 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.138+0000 7fb67c985700 1 Processor -- start 2024-03-21T12:46:04.846 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.138+0000 7fb67c985700 1 -- start start 2024-03-21T12:46:04.846 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.138+0000 7fb67c985700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb67819c3d0 con 0x7fb6781048b0 2024-03-21T12:46:04.846 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.139+0000 7fb67659c700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb6781048b0 0x7fb67819bcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41764/0 (socket says 172.21.15.43:41764) 2024-03-21T12:46:04.847 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.139+0000 7fb67659c700 1 -- 172.21.15.43:0/1696566992 learned_addr learned my addr 172.21.15.43:0/1696566992 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:04.847 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.139+0000 7fb66f7fe700 1 -- 172.21.15.43:0/1696566992 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4126301863 0 0) 0x7fb67819c3d0 con 0x7fb6781048b0 2024-03-21T12:46:04.847 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.139+0000 7fb66f7fe700 1 -- 172.21.15.43:0/1696566992 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6640055f0 con 0x7fb6781048b0 2024-03-21T12:46:04.847 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.139+0000 7fb66f7fe700 1 -- 172.21.15.43:0/1696566992 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2732442594 0 0) 0x7fb6640055f0 con 0x7fb6781048b0 2024-03-21T12:46:04.847 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.139+0000 7fb66f7fe700 1 -- 172.21.15.43:0/1696566992 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb67819c3d0 con 0x7fb6781048b0 2024-03-21T12:46:04.847 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.139+0000 7fb66f7fe700 1 -- 172.21.15.43:0/1696566992 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fb668001080 con 0x7fb6781048b0 2024-03-21T12:46:04.847 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.139+0000 7fb66f7fe700 1 -- 172.21.15.43:0/1696566992 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3074626800 0 0) 0x7fb67819c3d0 con 0x7fb6781048b0 2024-03-21T12:46:04.847 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.139+0000 7fb66f7fe700 1 -- 172.21.15.43:0/1696566992 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67819c5a0 con 0x7fb6781048b0 2024-03-21T12:46:04.847 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.139+0000 7fb67c985700 1 -- 172.21.15.43:0/1696566992 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67819c890 con 0x7fb6781048b0 2024-03-21T12:46:04.847 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.139+0000 7fb67c985700 1 -- 172.21.15.43:0/1696566992 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6781a03e0 con 0x7fb6781048b0 2024-03-21T12:46:04.847 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.139+0000 7fb66f7fe700 1 -- 172.21.15.43:0/1696566992 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7fb668001c80 con 0x7fb6781048b0 2024-03-21T12:46:04.847 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.140+0000 7fb66f7fe700 1 -- 172.21.15.43:0/1696566992 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fb6680057e0 con 0x7fb6781048b0 2024-03-21T12:46:04.847 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.140+0000 7fb66f7fe700 1 -- 172.21.15.43:0/1696566992 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 1) v1 ==== 807+0+0 (unknown 3189685207 0 0) 0x7fb668006220 con 0x7fb6781048b0 2024-03-21T12:46:04.848 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.140+0000 7fb66f7fe700 1 -- 172.21.15.43:0/1696566992 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 400852023 0 0) 0x7fb668007820 con 0x7fb6781048b0 2024-03-21T12:46:04.848 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.141+0000 7fb67c985700 1 -- 172.21.15.43:0/1696566992 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb658005150 con 0x7fb6781048b0 2024-03-21T12:46:04.848 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.143+0000 7fb66f7fe700 1 -- 172.21.15.43:0/1696566992 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7fb668004ab0 con 0x7fb6781048b0 2024-03-21T12:46:04.848 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.216+0000 7fb67c985700 1 -- 172.21.15.43:0/1696566992 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb658005440 con 0x7fb6781048b0 2024-03-21T12:46:04.848 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.217+0000 7fb66f7fe700 1 -- 172.21.15.43:0/1696566992 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (unknown 4201413639 0 4102693676) 0x7fb668004c90 con 0x7fb6781048b0 2024-03-21T12:46:04.848 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.218+0000 7fb67c985700 1 -- 172.21.15.43:0/1696566992 >> v1:172.21.15.43:6789/0 conn(0x7fb6781048b0 legacy=0x7fb67819bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:04.848 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.218+0000 7fb67c985700 1 -- 172.21.15.43:0/1696566992 shutdown_connections 2024-03-21T12:46:04.848 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.218+0000 7fb67c985700 1 -- 172.21.15.43:0/1696566992 >> 172.21.15.43:0/1696566992 conn(0x7fb678100470 msgr2=0x7fb6781028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:04.848 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.218+0000 7fb67c985700 1 -- 172.21.15.43:0/1696566992 shutdown_connections 2024-03-21T12:46:04.848 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:04.218+0000 7fb67c985700 1 -- 172.21.15.43:0/1696566992 wait complete. 2024-03-21T12:46:04.848 INFO:teuthology.orchestra.run.smithi043.stdout:mgr not available, waiting (1/15)... 2024-03-21T12:46:05.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:04 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:04.908+0000 7f35ae99a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T12:46:05.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:05 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:05.003+0000 7f35ae99a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T12:46:05.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:05 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:05.146+0000 7f35ae99a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T12:46:05.597 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:05 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:05.244+0000 7f35ae99a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T12:46:05.598 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:05 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:05.329+0000 7f35ae99a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T12:46:05.598 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:05 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:05.597+0000 7f35ae99a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T12:46:05.896 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:05 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:05.710+0000 7f35ae99a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T12:46:05.920 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:05 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:05.797+0000 7f35ae99a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T12:46:05.921 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:05 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:05.895+0000 7f35ae99a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T12:46:06.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:06 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:06.073+0000 7f35ae99a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T12:46:07.232 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:07 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:07.093+0000 7f35ae99a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T12:46:07.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:07 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:07.232+0000 7f35ae99a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T12:46:07.788 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:07 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2813463397' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T12:46:07.788 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:07 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:07.604+0000 7f35ae99a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T12:46:07.788 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:07 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:07.695+0000 7f35ae99a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T12:46:08.035 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-21T12:46:08.035 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-03-21T12:46:08.035 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "c06c8d1a-e780-11ee-95cd-87774f69a715", 2024-03-21T12:46:08.035 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T12:46:08.035 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T12:46:08.035 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T12:46:08.035 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T12:46:08.035 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:08.035 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T12:46:08.035 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T12:46:08.035 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-03-21T12:46:08.036 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-03-21T12:46:08.036 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T12:46:08.036 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T12:46:08.036 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-03-21T12:46:08.036 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-21T12:46:08.036 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T12:46:08.036 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T12:46:08.036 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T12:46:08.036 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T12:46:08.036 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:08.036 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T12:46:08.036 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T12:46:08.037 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T12:46:08.037 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T12:46:08.037 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T12:46:08.045 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T12:46:08.045 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T12:46:08.045 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T12:46:08.045 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:08.046 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T12:46:08.046 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T12:46:08.046 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T12:46:08.046 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T12:46:08.046 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T12:46:08.046 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T12:46:08.046 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T12:46:08.046 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T12:46:08.046 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T12:46:08.046 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:08.046 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T12:46:08.046 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T12:46:08.047 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T12:46:08.047 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T12:46:08.047 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:08.047 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T12:46:08.047 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-21T12:46:08.047 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T12:46:08.047 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T12:46:08.047 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T12:46:08.047 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T12:46:08.047 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T12:46:08.047 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-03-21T12:46:08.047 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T12:46:08.047 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:08.047 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T12:46:08.047 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T12:46:08.048 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T12:45:46.690987+0000", 2024-03-21T12:46:08.048 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T12:46:08.048 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:08.048 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T12:46:08.048 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-03-21T12:46:08.048 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.338+0000 7fc73974e700 1 Processor -- start 2024-03-21T12:46:08.048 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.338+0000 7fc73974e700 1 -- start start 2024-03-21T12:46:08.048 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.338+0000 7fc73974e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc7340796d0 con 0x7fc734106ab0 2024-03-21T12:46:08.048 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.338+0000 7fc732ffd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc734106ab0 0x7fc734108ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41776/0 (socket says 172.21.15.43:41776) 2024-03-21T12:46:08.048 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.338+0000 7fc732ffd700 1 -- 172.21.15.43:0/761061357 learned_addr learned my addr 172.21.15.43:0/761061357 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:08.048 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.339+0000 7fc731ffb700 1 -- 172.21.15.43:0/761061357 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3133745242 0 0) 0x7fc7340796d0 con 0x7fc734106ab0 2024-03-21T12:46:08.048 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.339+0000 7fc731ffb700 1 -- 172.21.15.43:0/761061357 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc71c0035b0 con 0x7fc734106ab0 2024-03-21T12:46:08.049 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.339+0000 7fc731ffb700 1 -- 172.21.15.43:0/761061357 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2453637060 0 0) 0x7fc71c0035b0 con 0x7fc734106ab0 2024-03-21T12:46:08.049 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.339+0000 7fc731ffb700 1 -- 172.21.15.43:0/761061357 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73410b5d0 con 0x7fc734106ab0 2024-03-21T12:46:08.049 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.339+0000 7fc731ffb700 1 -- 172.21.15.43:0/761061357 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fc724001580 con 0x7fc734106ab0 2024-03-21T12:46:08.049 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.339+0000 7fc731ffb700 1 -- 172.21.15.43:0/761061357 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7fc724001d20 con 0x7fc734106ab0 2024-03-21T12:46:08.049 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.339+0000 7fc731ffb700 1 -- 172.21.15.43:0/761061357 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fc724005960 con 0x7fc734106ab0 2024-03-21T12:46:08.049 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.340+0000 7fc73974e700 1 -- 172.21.15.43:0/761061357 >> v1:172.21.15.43:6789/0 conn(0x7fc734106ab0 legacy=0x7fc734108ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:08.049 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.340+0000 7fc73974e700 1 -- 172.21.15.43:0/761061357 shutdown_connections 2024-03-21T12:46:08.049 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.340+0000 7fc73974e700 1 -- 172.21.15.43:0/761061357 >> 172.21.15.43:0/761061357 conn(0x7fc7341004c0 msgr2=0x7fc7341028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:08.049 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.340+0000 7fc73974e700 1 -- 172.21.15.43:0/761061357 shutdown_connections 2024-03-21T12:46:08.049 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.340+0000 7fc73974e700 1 -- 172.21.15.43:0/761061357 wait complete. 2024-03-21T12:46:08.049 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.340+0000 7fc73974e700 1 Processor -- start 2024-03-21T12:46:08.049 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.340+0000 7fc73974e700 1 -- start start 2024-03-21T12:46:08.049 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.341+0000 7fc73974e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc73419a1a0 con 0x7fc734106ab0 2024-03-21T12:46:08.049 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.341+0000 7fc732ffd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc734106ab0 0x7fc734199a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41792/0 (socket says 172.21.15.43:41792) 2024-03-21T12:46:08.050 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.341+0000 7fc732ffd700 1 -- 172.21.15.43:0/2813463397 learned_addr learned my addr 172.21.15.43:0/2813463397 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:08.050 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.341+0000 7fc71bfff700 1 -- 172.21.15.43:0/2813463397 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3049758347 0 0) 0x7fc73419a1a0 con 0x7fc734106ab0 2024-03-21T12:46:08.050 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.341+0000 7fc71bfff700 1 -- 172.21.15.43:0/2813463397 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc71c0055f0 con 0x7fc734106ab0 2024-03-21T12:46:08.050 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.341+0000 7fc71bfff700 1 -- 172.21.15.43:0/2813463397 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3151582719 0 0) 0x7fc71c0055f0 con 0x7fc734106ab0 2024-03-21T12:46:08.050 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.341+0000 7fc71bfff700 1 -- 172.21.15.43:0/2813463397 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc73419a1a0 con 0x7fc734106ab0 2024-03-21T12:46:08.050 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.341+0000 7fc71bfff700 1 -- 172.21.15.43:0/2813463397 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fc724004870 con 0x7fc734106ab0 2024-03-21T12:46:08.050 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.341+0000 7fc71bfff700 1 -- 172.21.15.43:0/2813463397 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2354153772 0 0) 0x7fc73419a1a0 con 0x7fc734106ab0 2024-03-21T12:46:08.050 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.341+0000 7fc71bfff700 1 -- 172.21.15.43:0/2813463397 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc73419a370 con 0x7fc734106ab0 2024-03-21T12:46:08.050 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.341+0000 7fc73974e700 1 -- 172.21.15.43:0/2813463397 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc73419a660 con 0x7fc734106ab0 2024-03-21T12:46:08.050 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.342+0000 7fc73974e700 1 -- 172.21.15.43:0/2813463397 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc73419e1b0 con 0x7fc734106ab0 2024-03-21T12:46:08.050 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.342+0000 7fc71bfff700 1 -- 172.21.15.43:0/2813463397 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7fc724001190 con 0x7fc734106ab0 2024-03-21T12:46:08.050 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.342+0000 7fc71bfff700 1 -- 172.21.15.43:0/2813463397 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fc724005f70 con 0x7fc734106ab0 2024-03-21T12:46:08.050 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.342+0000 7fc71bfff700 1 -- 172.21.15.43:0/2813463397 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 1) v1 ==== 807+0+0 (unknown 3189685207 0 0) 0x7fc724006560 con 0x7fc734106ab0 2024-03-21T12:46:08.050 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.342+0000 7fc71bfff700 1 -- 172.21.15.43:0/2813463397 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 400852023 0 0) 0x7fc724007c50 con 0x7fc734106ab0 2024-03-21T12:46:08.051 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.343+0000 7fc73974e700 1 -- 172.21.15.43:0/2813463397 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc720005150 con 0x7fc734106ab0 2024-03-21T12:46:08.051 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.345+0000 7fc71bfff700 1 -- 172.21.15.43:0/2813463397 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7fc724001c00 con 0x7fc734106ab0 2024-03-21T12:46:08.051 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.434+0000 7fc73974e700 1 -- 172.21.15.43:0/2813463397 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc720005440 con 0x7fc734106ab0 2024-03-21T12:46:08.051 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.435+0000 7fc71bfff700 1 -- 172.21.15.43:0/2813463397 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (unknown 4201413639 0 2601183634) 0x7fc724007f30 con 0x7fc734106ab0 2024-03-21T12:46:08.051 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.436+0000 7fc73974e700 1 -- 172.21.15.43:0/2813463397 >> v1:172.21.15.43:6789/0 conn(0x7fc734106ab0 legacy=0x7fc734199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:08.051 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.436+0000 7fc73974e700 1 -- 172.21.15.43:0/2813463397 shutdown_connections 2024-03-21T12:46:08.051 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.436+0000 7fc73974e700 1 -- 172.21.15.43:0/2813463397 >> 172.21.15.43:0/2813463397 conn(0x7fc7341004c0 msgr2=0x7fc734190800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:08.051 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.436+0000 7fc73974e700 1 -- 172.21.15.43:0/2813463397 shutdown_connections 2024-03-21T12:46:08.051 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:07.436+0000 7fc73974e700 1 -- 172.21.15.43:0/2813463397 wait complete. 2024-03-21T12:46:08.051 INFO:teuthology.orchestra.run.smithi043.stdout:mgr not available, waiting (2/15)... 2024-03-21T12:46:08.091 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:07 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:07.899+0000 7f35ae99a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T12:46:08.383 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:08.091+0000 7f35ae99a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T12:46:08.643 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:08.383+0000 7f35ae99a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T12:46:08.643 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:08.643+0000 7f35ae99a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T12:46:08.983 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:08.884+0000 7f35ae99a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T12:46:09.983 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:09 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:09.708+0000 7f35ae99a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T12:46:10.430 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:10 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:10.327+0000 7f35ae99a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T12:46:10.430 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:10 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:10.430+0000 7f35ae99a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T12:46:10.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1591163400' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T12:46:10.734 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:10 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:10.666+0000 7f35ae99a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T12:46:11.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-21T12:46:11.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-03-21T12:46:11.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "c06c8d1a-e780-11ee-95cd-87774f69a715", 2024-03-21T12:46:11.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T12:46:11.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T12:46:11.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T12:46:11.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T12:46:11.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:11.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T12:46:11.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T12:46:11.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-03-21T12:46:11.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-03-21T12:46:11.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T12:46:11.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T12:46:11.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-03-21T12:46:11.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-03-21T12:46:11.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T12:46:11.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T12:46:11.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T12:46:11.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T12:46:11.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:11.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T12:46:11.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T12:46:11.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T12:46:11.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T12:46:11.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T12:46:11.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T12:46:11.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T12:46:11.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T12:46:11.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:11.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T12:46:11.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T12:46:11.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T12:46:11.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T12:46:11.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T12:46:11.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T12:46:11.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T12:46:11.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T12:46:11.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T12:46:11.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:11.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T12:46:11.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T12:46:11.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T12:46:11.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T12:46:11.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:11.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T12:46:11.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-21T12:46:11.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T12:46:11.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T12:46:11.142 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T12:46:11.142 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T12:46:11.142 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T12:46:11.142 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-03-21T12:46:11.142 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T12:46:11.142 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:11.142 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T12:46:11.142 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T12:46:11.142 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T12:45:46.690987+0000", 2024-03-21T12:46:11.142 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T12:46:11.142 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:11.143 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T12:46:11.143 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-03-21T12:46:11.143 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.459+0000 7fe1569a4700 1 Processor -- start 2024-03-21T12:46:11.143 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.459+0000 7fe1569a4700 1 -- start start 2024-03-21T12:46:11.143 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.459+0000 7fe1569a4700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe150079fe0 con 0x7fe150106a40 2024-03-21T12:46:11.143 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.460+0000 7fe14ffff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe150106a40 0x7fe150108e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41800/0 (socket says 172.21.15.43:41800) 2024-03-21T12:46:11.143 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.460+0000 7fe14ffff700 1 -- 172.21.15.43:0/1654063130 learned_addr learned my addr 172.21.15.43:0/1654063130 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:11.143 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.460+0000 7fe14effd700 1 -- 172.21.15.43:0/1654063130 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2987806676 0 0) 0x7fe150079fe0 con 0x7fe150106a40 2024-03-21T12:46:11.143 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.460+0000 7fe14effd700 1 -- 172.21.15.43:0/1654063130 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe13c0035b0 con 0x7fe150106a40 2024-03-21T12:46:11.143 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.460+0000 7fe14effd700 1 -- 172.21.15.43:0/1654063130 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3960275875 0 0) 0x7fe13c0035b0 con 0x7fe150106a40 2024-03-21T12:46:11.143 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.460+0000 7fe14effd700 1 -- 172.21.15.43:0/1654063130 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15010b560 con 0x7fe150106a40 2024-03-21T12:46:11.143 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.460+0000 7fe14effd700 1 -- 172.21.15.43:0/1654063130 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fe140001580 con 0x7fe150106a40 2024-03-21T12:46:11.143 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.460+0000 7fe14effd700 1 -- 172.21.15.43:0/1654063130 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7fe140001d20 con 0x7fe150106a40 2024-03-21T12:46:11.143 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.460+0000 7fe14effd700 1 -- 172.21.15.43:0/1654063130 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fe140005960 con 0x7fe150106a40 2024-03-21T12:46:11.143 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.461+0000 7fe1569a4700 1 -- 172.21.15.43:0/1654063130 >> v1:172.21.15.43:6789/0 conn(0x7fe150106a40 legacy=0x7fe150108e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:11.143 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.461+0000 7fe1569a4700 1 -- 172.21.15.43:0/1654063130 shutdown_connections 2024-03-21T12:46:11.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.461+0000 7fe1569a4700 1 -- 172.21.15.43:0/1654063130 >> 172.21.15.43:0/1654063130 conn(0x7fe150100470 msgr2=0x7fe1501028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:11.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.461+0000 7fe1569a4700 1 -- 172.21.15.43:0/1654063130 shutdown_connections 2024-03-21T12:46:11.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.461+0000 7fe1569a4700 1 -- 172.21.15.43:0/1654063130 wait complete. 2024-03-21T12:46:11.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.462+0000 7fe1569a4700 1 Processor -- start 2024-03-21T12:46:11.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.462+0000 7fe1569a4700 1 -- start start 2024-03-21T12:46:11.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.462+0000 7fe1569a4700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe15019a200 con 0x7fe150106a40 2024-03-21T12:46:11.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.462+0000 7fe14ffff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe150106a40 0x7fe150199af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41810/0 (socket says 172.21.15.43:41810) 2024-03-21T12:46:11.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.462+0000 7fe14ffff700 1 -- 172.21.15.43:0/1591163400 learned_addr learned my addr 172.21.15.43:0/1591163400 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:11.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.462+0000 7fe14d7fa700 1 -- 172.21.15.43:0/1591163400 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1541843056 0 0) 0x7fe15019a200 con 0x7fe150106a40 2024-03-21T12:46:11.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.462+0000 7fe14d7fa700 1 -- 172.21.15.43:0/1591163400 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe13c0055f0 con 0x7fe150106a40 2024-03-21T12:46:11.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.463+0000 7fe14d7fa700 1 -- 172.21.15.43:0/1591163400 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1855956338 0 0) 0x7fe13c0055f0 con 0x7fe150106a40 2024-03-21T12:46:11.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.463+0000 7fe14d7fa700 1 -- 172.21.15.43:0/1591163400 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe15019a200 con 0x7fe150106a40 2024-03-21T12:46:11.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.463+0000 7fe14d7fa700 1 -- 172.21.15.43:0/1591163400 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fe140001080 con 0x7fe150106a40 2024-03-21T12:46:11.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.463+0000 7fe14d7fa700 1 -- 172.21.15.43:0/1591163400 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3663641631 0 0) 0x7fe15019a200 con 0x7fe150106a40 2024-03-21T12:46:11.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.463+0000 7fe14d7fa700 1 -- 172.21.15.43:0/1591163400 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15019a3d0 con 0x7fe150106a40 2024-03-21T12:46:11.144 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.463+0000 7fe1569a4700 1 -- 172.21.15.43:0/1591163400 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe15019a6c0 con 0x7fe150106a40 2024-03-21T12:46:11.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.463+0000 7fe1569a4700 1 -- 172.21.15.43:0/1591163400 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe15019e210 con 0x7fe150106a40 2024-03-21T12:46:11.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.463+0000 7fe14d7fa700 1 -- 172.21.15.43:0/1591163400 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7fe140004ab0 con 0x7fe150106a40 2024-03-21T12:46:11.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.463+0000 7fe14d7fa700 1 -- 172.21.15.43:0/1591163400 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fe1400061d0 con 0x7fe150106a40 2024-03-21T12:46:11.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.463+0000 7fe14d7fa700 1 -- 172.21.15.43:0/1591163400 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 1) v1 ==== 807+0+0 (unknown 3189685207 0 0) 0x7fe140006820 con 0x7fe150106a40 2024-03-21T12:46:11.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.463+0000 7fe14d7fa700 1 -- 172.21.15.43:0/1591163400 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 400852023 0 0) 0x7fe140007ce0 con 0x7fe150106a40 2024-03-21T12:46:11.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.464+0000 7fe1569a4700 1 -- 172.21.15.43:0/1591163400 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe15010b0c0 con 0x7fe150106a40 2024-03-21T12:46:11.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.466+0000 7fe14d7fa700 1 -- 172.21.15.43:0/1591163400 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (unknown 1092875540 0 2031569661) 0x7fe1400056d0 con 0x7fe150106a40 2024-03-21T12:46:11.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.540+0000 7fe1569a4700 1 -- 172.21.15.43:0/1591163400 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fe15002db70 con 0x7fe150106a40 2024-03-21T12:46:11.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.540+0000 7fe14d7fa700 1 -- 172.21.15.43:0/1591163400 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (unknown 4201413639 0 3654985111) 0x7fe140005950 con 0x7fe150106a40 2024-03-21T12:46:11.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.541+0000 7fe1569a4700 1 -- 172.21.15.43:0/1591163400 >> v1:172.21.15.43:6789/0 conn(0x7fe150106a40 legacy=0x7fe150199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:11.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.541+0000 7fe1569a4700 1 -- 172.21.15.43:0/1591163400 shutdown_connections 2024-03-21T12:46:11.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.541+0000 7fe1569a4700 1 -- 172.21.15.43:0/1591163400 >> 172.21.15.43:0/1591163400 conn(0x7fe150100470 msgr2=0x7fe150190810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:11.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.541+0000 7fe1569a4700 1 -- 172.21.15.43:0/1591163400 shutdown_connections 2024-03-21T12:46:11.145 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:10.542+0000 7fe1569a4700 1 -- 172.21.15.43:0/1591163400 wait complete. 2024-03-21T12:46:11.146 INFO:teuthology.orchestra.run.smithi043.stdout:mgr not available, waiting (3/15)... 2024-03-21T12:46:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:11 smithi043 ceph-mon[30062]: Activating manager daemon y 2024-03-21T12:46:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:11 smithi043 ceph-mon[30062]: mgrmap e2: y(active, starting, since 0.00301287s) 2024-03-21T12:46:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:11 smithi043 ceph-mon[30062]: from='mgr.14100 v1:172.21.15.43:0/1326076996' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T12:46:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:11 smithi043 ceph-mon[30062]: from='mgr.14100 v1:172.21.15.43:0/1326076996' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T12:46:11.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:11 smithi043 ceph-mon[30062]: from='mgr.14100 v1:172.21.15.43:0/1326076996' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T12:46:11.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:11 smithi043 ceph-mon[30062]: from='mgr.14100 v1:172.21.15.43:0/1326076996' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T12:46:11.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:11 smithi043 ceph-mon[30062]: from='mgr.14100 v1:172.21.15.43:0/1326076996' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T12:46:11.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:11 smithi043 ceph-mon[30062]: from='mgr.14100 v1:172.21.15.43:0/1326076996' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T12:46:11.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:11 smithi043 ceph-mon[30062]: from='mgr.14100 v1:172.21.15.43:0/1326076996' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T12:46:11.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:11 smithi043 ceph-mon[30062]: from='mgr.14100 v1:172.21.15.43:0/1326076996' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T12:46:11.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:11 smithi043 ceph-mon[30062]: Manager daemon y is now available 2024-03-21T12:46:11.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:11 smithi043 ceph-mon[30062]: from='mgr.14100 v1:172.21.15.43:0/1326076996' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-21T12:46:11.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:11 smithi043 ceph-mon[30062]: from='mgr.14100 v1:172.21.15.43:0/1326076996' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-21T12:46:11.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:11 smithi043 ceph-mon[30062]: from='mgr.14100 v1:172.21.15.43:0/1326076996' entity='mgr.y' 2024-03-21T12:46:11.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:11 smithi043 ceph-mon[30062]: from='mgr.14100 v1:172.21.15.43:0/1326076996' entity='mgr.y' 2024-03-21T12:46:11.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:11 smithi043 ceph-mon[30062]: from='mgr.14100 v1:172.21.15.43:0/1326076996' entity='mgr.y' 2024-03-21T12:46:12.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:12 smithi043 ceph-mon[30062]: mgrmap e3: y(active, since 1.00713s) 2024-03-21T12:46:14.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-21T12:46:14.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-03-21T12:46:14.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsid": "c06c8d1a-e780-11ee-95cd-87774f69a715", 2024-03-21T12:46:14.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T12:46:14.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T12:46:14.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T12:46:14.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T12:46:14.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:14.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T12:46:14.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T12:46:14.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 0 2024-03-21T12:46:14.488 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-03-21T12:46:14.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T12:46:14.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T12:46:14.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-03-21T12:46:14.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-03-21T12:46:14.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T12:46:14.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T12:46:14.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T12:46:14.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T12:46:14.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:14.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T12:46:14.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T12:46:14.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T12:46:14.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T12:46:14.489 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T12:46:14.497 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T12:46:14.497 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T12:46:14.497 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T12:46:14.497 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:14.497 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T12:46:14.497 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T12:46:14.497 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T12:46:14.497 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T12:46:14.498 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T12:46:14.498 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T12:46:14.498 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T12:46:14.498 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T12:46:14.498 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T12:46:14.498 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:14.498 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T12:46:14.498 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T12:46:14.498 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T12:46:14.498 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T12:46:14.498 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:14.498 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T12:46:14.498 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-21T12:46:14.498 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T12:46:14.499 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T12:46:14.499 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T12:46:14.499 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T12:46:14.499 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T12:46:14.499 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ], 2024-03-21T12:46:14.499 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T12:46:14.499 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:14.499 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T12:46:14.499 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T12:46:14.499 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T12:45:46.690987+0000", 2024-03-21T12:46:14.499 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T12:46:14.499 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout }, 2024-03-21T12:46:14.499 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T12:46:14.499 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-03-21T12:46:14.499 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.589+0000 7f11c6e30700 1 Processor -- start 2024-03-21T12:46:14.499 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.589+0000 7f11c6e30700 1 -- start start 2024-03-21T12:46:14.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.590+0000 7f11c6e30700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f11c0109320 con 0x7f11c0104e70 2024-03-21T12:46:14.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.590+0000 7f11c4bcc700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f11c0104e70 0x7f11c0105290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41468/0 (socket says 172.21.15.43:41468) 2024-03-21T12:46:14.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.590+0000 7f11c4bcc700 1 -- 172.21.15.43:0/2801349243 learned_addr learned my addr 172.21.15.43:0/2801349243 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:14.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.590+0000 7f11bf7fe700 1 -- 172.21.15.43:0/2801349243 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3035904767 0 0) 0x7f11c0109320 con 0x7f11c0104e70 2024-03-21T12:46:14.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.590+0000 7f11bf7fe700 1 -- 172.21.15.43:0/2801349243 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11b00035b0 con 0x7f11c0104e70 2024-03-21T12:46:14.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.590+0000 7f11bf7fe700 1 -- 172.21.15.43:0/2801349243 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1371438091 0 0) 0x7f11b00035b0 con 0x7f11c0104e70 2024-03-21T12:46:14.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.590+0000 7f11bf7fe700 1 -- 172.21.15.43:0/2801349243 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c010a500 con 0x7f11c0104e70 2024-03-21T12:46:14.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.590+0000 7f11bf7fe700 1 -- 172.21.15.43:0/2801349243 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f11b4001580 con 0x7f11c0104e70 2024-03-21T12:46:14.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.591+0000 7f11bf7fe700 1 -- 172.21.15.43:0/2801349243 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7f11b4001c50 con 0x7f11c0104e70 2024-03-21T12:46:14.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.591+0000 7f11bf7fe700 1 -- 172.21.15.43:0/2801349243 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f11b4005960 con 0x7f11c0104e70 2024-03-21T12:46:14.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.591+0000 7f11c6e30700 1 -- 172.21.15.43:0/2801349243 >> v1:172.21.15.43:6789/0 conn(0x7f11c0104e70 legacy=0x7f11c0105290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:14.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.591+0000 7f11c6e30700 1 -- 172.21.15.43:0/2801349243 shutdown_connections 2024-03-21T12:46:14.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.591+0000 7f11c6e30700 1 -- 172.21.15.43:0/2801349243 >> 172.21.15.43:0/2801349243 conn(0x7f11c01004c0 msgr2=0x7f11c01028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:14.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.591+0000 7f11c6e30700 1 -- 172.21.15.43:0/2801349243 shutdown_connections 2024-03-21T12:46:14.500 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.591+0000 7f11c6e30700 1 -- 172.21.15.43:0/2801349243 wait complete. 2024-03-21T12:46:14.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.592+0000 7f11c6e30700 1 Processor -- start 2024-03-21T12:46:14.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.592+0000 7f11c6e30700 1 -- start start 2024-03-21T12:46:14.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.592+0000 7f11c6e30700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f11c0079350 con 0x7f11c0104e70 2024-03-21T12:46:14.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.592+0000 7f11c4bcc700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f11c0104e70 0x7f11c019c7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41470/0 (socket says 172.21.15.43:41470) 2024-03-21T12:46:14.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.592+0000 7f11c4bcc700 1 -- 172.21.15.43:0/1726996684 learned_addr learned my addr 172.21.15.43:0/1726996684 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:14.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.592+0000 7f11bdffb700 1 -- 172.21.15.43:0/1726996684 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1244330101 0 0) 0x7f11c0079350 con 0x7f11c0104e70 2024-03-21T12:46:14.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.592+0000 7f11bdffb700 1 -- 172.21.15.43:0/1726996684 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11b0005640 con 0x7f11c0104e70 2024-03-21T12:46:14.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.592+0000 7f11bdffb700 1 -- 172.21.15.43:0/1726996684 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 385685966 0 0) 0x7f11b0005640 con 0x7f11c0104e70 2024-03-21T12:46:14.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.592+0000 7f11bdffb700 1 -- 172.21.15.43:0/1726996684 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f11c0079350 con 0x7f11c0104e70 2024-03-21T12:46:14.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.592+0000 7f11bdffb700 1 -- 172.21.15.43:0/1726996684 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f11b4001080 con 0x7f11c0104e70 2024-03-21T12:46:14.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.593+0000 7f11bdffb700 1 -- 172.21.15.43:0/1726996684 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2512632134 0 0) 0x7f11c0079350 con 0x7f11c0104e70 2024-03-21T12:46:14.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.593+0000 7f11bdffb700 1 -- 172.21.15.43:0/1726996684 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c0079520 con 0x7f11c0104e70 2024-03-21T12:46:14.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.593+0000 7f11c6e30700 1 -- 172.21.15.43:0/1726996684 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c019ded0 con 0x7f11c0104e70 2024-03-21T12:46:14.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.593+0000 7f11bdffb700 1 -- 172.21.15.43:0/1726996684 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7f11b40049d0 con 0x7f11c0104e70 2024-03-21T12:46:14.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.593+0000 7f11bdffb700 1 -- 172.21.15.43:0/1726996684 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f11b40062a0 con 0x7f11c0104e70 2024-03-21T12:46:14.501 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.593+0000 7f11c6e30700 1 -- 172.21.15.43:0/1726996684 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c019e270 con 0x7f11c0104e70 2024-03-21T12:46:14.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.593+0000 7f11bdffb700 1 -- 172.21.15.43:0/1726996684 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 3) v1 ==== 46142+0+0 (unknown 1302585846 0 0) 0x7f11b4011a00 con 0x7f11c0104e70 2024-03-21T12:46:14.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.594+0000 7f11bdffb700 1 -- 172.21.15.43:0/1726996684 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 400852023 0 0) 0x7f11b4047680 con 0x7f11c0104e70 2024-03-21T12:46:14.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.594+0000 7f11c6e30700 1 -- 172.21.15.43:0/1726996684 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c00637c0 con 0x7f11c0104e70 2024-03-21T12:46:14.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.599+0000 7f11bdffb700 1 -- 172.21.15.43:0/1726996684 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f11b4017540 con 0x7f11c0104e70 2024-03-21T12:46:14.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.684+0000 7f11bdffb700 1 -- 172.21.15.43:0/1726996684 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mgrmap(e 4) v1 ==== 46248+0+0 (unknown 3190036632 0 0) 0x7f11b4012f80 con 0x7f11c0104e70 2024-03-21T12:46:14.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.889+0000 7f11c6e30700 1 -- 172.21.15.43:0/1726996684 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f11c002dad0 con 0x7f11c0104e70 2024-03-21T12:46:14.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.889+0000 7f11bdffb700 1 -- 172.21.15.43:0/1726996684 <== mon.0 v1:172.21.15.43:6789/0 11 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (unknown 4201413639 0 2807407381) 0x7f11b4005bf0 con 0x7f11c0104e70 2024-03-21T12:46:14.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.891+0000 7f11c6e30700 1 -- 172.21.15.43:0/1726996684 >> v1:172.21.15.43:6800/2056453291 conn(0x7f11b003a4e0 legacy=0x7f11b003c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:14.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.891+0000 7f11c6e30700 1 -- 172.21.15.43:0/1726996684 >> v1:172.21.15.43:6789/0 conn(0x7f11c0104e70 legacy=0x7f11c019c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:14.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.891+0000 7f11c6e30700 1 -- 172.21.15.43:0/1726996684 shutdown_connections 2024-03-21T12:46:14.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.891+0000 7f11c6e30700 1 -- 172.21.15.43:0/1726996684 >> 172.21.15.43:0/1726996684 conn(0x7f11c01004c0 msgr2=0x7f11c01028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:14.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.892+0000 7f11c6e30700 1 -- 172.21.15.43:0/1726996684 shutdown_connections 2024-03-21T12:46:14.502 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:13.892+0000 7f11c6e30700 1 -- 172.21.15.43:0/1726996684 wait complete. 2024-03-21T12:46:14.502 INFO:teuthology.orchestra.run.smithi043.stdout:mgr is available 2024-03-21T12:46:14.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:14 smithi043 ceph-mon[30062]: mgrmap e4: y(active, since 3s) 2024-03-21T12:46:14.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:14 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1726996684' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T12:46:15.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:15 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/71829408' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-21T12:46:15.797 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-21T12:46:15.797 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [global] 2024-03-21T12:46:15.797 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout fsid = c06c8d1a-e780-11ee-95cd-87774f69a715 2024-03-21T12:46:15.797 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-21T12:46:15.797 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-21T12:46:15.797 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-21T12:46:15.797 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-21T12:46:15.797 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-21T12:46:15.797 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-21T12:46:15.797 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-21T12:46:15.798 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 2024-03-21T12:46:15.798 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout [osd] 2024-03-21T12:46:15.798 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-21T12:46:15.798 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-21T12:46:15.798 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.966+0000 7ff9f3a45700 1 Processor -- start 2024-03-21T12:46:15.798 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.966+0000 7ff9f3a45700 1 -- start start 2024-03-21T12:46:15.798 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.966+0000 7ff9f3a45700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff9ec10a410 con 0x7ff9ec105fe0 2024-03-21T12:46:15.798 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.966+0000 7ff9f17e1700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff9ec105fe0 0x7ff9ec106400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41472/0 (socket says 172.21.15.43:41472) 2024-03-21T12:46:15.798 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.966+0000 7ff9f17e1700 1 -- 172.21.15.43:0/2805256064 learned_addr learned my addr 172.21.15.43:0/2805256064 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:15.798 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.966+0000 7ff9e3fff700 1 -- 172.21.15.43:0/2805256064 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2002908788 0 0) 0x7ff9ec10a410 con 0x7ff9ec105fe0 2024-03-21T12:46:15.798 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.966+0000 7ff9e3fff700 1 -- 172.21.15.43:0/2805256064 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9d80035b0 con 0x7ff9ec105fe0 2024-03-21T12:46:15.798 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.967+0000 7ff9e3fff700 1 -- 172.21.15.43:0/2805256064 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2463349025 0 0) 0x7ff9d80035b0 con 0x7ff9ec105fe0 2024-03-21T12:46:15.798 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.967+0000 7ff9e3fff700 1 -- 172.21.15.43:0/2805256064 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ec10b5f0 con 0x7ff9ec105fe0 2024-03-21T12:46:15.799 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.967+0000 7ff9e3fff700 1 -- 172.21.15.43:0/2805256064 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7ff9e8001580 con 0x7ff9ec105fe0 2024-03-21T12:46:15.799 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.967+0000 7ff9e3fff700 1 -- 172.21.15.43:0/2805256064 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7ff9e8001d20 con 0x7ff9ec105fe0 2024-03-21T12:46:15.799 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.967+0000 7ff9e3fff700 1 -- 172.21.15.43:0/2805256064 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7ff9e8005960 con 0x7ff9ec105fe0 2024-03-21T12:46:15.799 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.967+0000 7ff9f3a45700 1 -- 172.21.15.43:0/2805256064 >> v1:172.21.15.43:6789/0 conn(0x7ff9ec105fe0 legacy=0x7ff9ec106400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:15.799 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.967+0000 7ff9f3a45700 1 -- 172.21.15.43:0/2805256064 shutdown_connections 2024-03-21T12:46:15.799 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.967+0000 7ff9f3a45700 1 -- 172.21.15.43:0/2805256064 >> 172.21.15.43:0/2805256064 conn(0x7ff9ec1015a0 msgr2=0x7ff9ec1039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:15.799 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.968+0000 7ff9f3a45700 1 -- 172.21.15.43:0/2805256064 shutdown_connections 2024-03-21T12:46:15.799 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.968+0000 7ff9f3a45700 1 -- 172.21.15.43:0/2805256064 wait complete. 2024-03-21T12:46:15.799 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.968+0000 7ff9f3a45700 1 Processor -- start 2024-03-21T12:46:15.799 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.968+0000 7ff9f3a45700 1 -- start start 2024-03-21T12:46:15.799 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.968+0000 7ff9f3a45700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff9ec19fb00 con 0x7ff9ec105fe0 2024-03-21T12:46:15.799 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.968+0000 7ff9f17e1700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff9ec105fe0 0x7ff9ec07bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41480/0 (socket says 172.21.15.43:41480) 2024-03-21T12:46:15.799 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.968+0000 7ff9f17e1700 1 -- 172.21.15.43:0/71829408 learned_addr learned my addr 172.21.15.43:0/71829408 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:15.799 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.969+0000 7ff9e27fc700 1 -- 172.21.15.43:0/71829408 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3508171679 0 0) 0x7ff9ec19fb00 con 0x7ff9ec105fe0 2024-03-21T12:46:15.800 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.969+0000 7ff9e27fc700 1 -- 172.21.15.43:0/71829408 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9d80055f0 con 0x7ff9ec105fe0 2024-03-21T12:46:15.800 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.969+0000 7ff9e27fc700 1 -- 172.21.15.43:0/71829408 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1393876346 0 0) 0x7ff9d80055f0 con 0x7ff9ec105fe0 2024-03-21T12:46:15.800 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.969+0000 7ff9e27fc700 1 -- 172.21.15.43:0/71829408 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff9ec19fb00 con 0x7ff9ec105fe0 2024-03-21T12:46:15.800 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.969+0000 7ff9e27fc700 1 -- 172.21.15.43:0/71829408 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7ff9e8004870 con 0x7ff9ec105fe0 2024-03-21T12:46:15.800 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.969+0000 7ff9e27fc700 1 -- 172.21.15.43:0/71829408 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 830764964 0 0) 0x7ff9ec19fb00 con 0x7ff9ec105fe0 2024-03-21T12:46:15.800 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.969+0000 7ff9e27fc700 1 -- 172.21.15.43:0/71829408 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ec1a0ce0 con 0x7ff9ec105fe0 2024-03-21T12:46:15.800 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.969+0000 7ff9f3a45700 1 -- 172.21.15.43:0/71829408 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9ec19fd30 con 0x7ff9ec105fe0 2024-03-21T12:46:15.800 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.969+0000 7ff9f3a45700 1 -- 172.21.15.43:0/71829408 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ec1a0270 con 0x7ff9ec105fe0 2024-03-21T12:46:15.800 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.969+0000 7ff9e27fc700 1 -- 172.21.15.43:0/71829408 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7ff9e8001190 con 0x7ff9ec105fe0 2024-03-21T12:46:15.800 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.969+0000 7ff9e27fc700 1 -- 172.21.15.43:0/71829408 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7ff9e8005f60 con 0x7ff9ec105fe0 2024-03-21T12:46:15.800 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.970+0000 7ff9e27fc700 1 -- 172.21.15.43:0/71829408 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 4) v1 ==== 46248+0+0 (unknown 3190036632 0 0) 0x7ff9e80115b0 con 0x7ff9ec105fe0 2024-03-21T12:46:15.800 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.970+0000 7ff9f3a45700 1 -- 172.21.15.43:0/71829408 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9d0005150 con 0x7ff9ec105fe0 2024-03-21T12:46:15.800 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.970+0000 7ff9e27fc700 1 -- 172.21.15.43:0/71829408 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 400852023 0 0) 0x7ff9e8047880 con 0x7ff9ec105fe0 2024-03-21T12:46:15.800 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:14.975+0000 7ff9e27fc700 1 -- 172.21.15.43:0/71829408 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7ff9e8001cc0 con 0x7ff9ec105fe0 2024-03-21T12:46:15.800 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:15.177+0000 7ff9f3a45700 1 -- 172.21.15.43:0/71829408 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7ff9d0002430 con 0x7ff9ec105fe0 2024-03-21T12:46:15.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:15.177+0000 7ff9e27fc700 1 -- 172.21.15.43:0/71829408 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+282 (unknown 1187553405 0 175727616) 0x7ff9e8017490 con 0x7ff9ec105fe0 2024-03-21T12:46:15.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:15.179+0000 7ff9f3a45700 1 -- 172.21.15.43:0/71829408 >> v1:172.21.15.43:6800/2056453291 conn(0x7ff9d803a4c0 legacy=0x7ff9d803c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:15.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:15.179+0000 7ff9f3a45700 1 -- 172.21.15.43:0/71829408 >> v1:172.21.15.43:6789/0 conn(0x7ff9ec105fe0 legacy=0x7ff9ec07bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:15.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:15.180+0000 7ff9f3a45700 1 -- 172.21.15.43:0/71829408 shutdown_connections 2024-03-21T12:46:15.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:15.180+0000 7ff9f3a45700 1 -- 172.21.15.43:0/71829408 >> 172.21.15.43:0/71829408 conn(0x7ff9ec1015a0 msgr2=0x7ff9ec103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:15.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:15.180+0000 7ff9f3a45700 1 -- 172.21.15.43:0/71829408 shutdown_connections 2024-03-21T12:46:15.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:15.180+0000 7ff9f3a45700 1 -- 172.21.15.43:0/71829408 wait complete. 2024-03-21T12:46:15.801 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling cephadm module... 2024-03-21T12:46:16.979 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:16 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3148790555' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-21T12:46:16.979 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:16 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ignoring --setuser ceph since I am not root 2024-03-21T12:46:16.979 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:16 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ignoring --setgroup ceph since I am not root 2024-03-21T12:46:17.279 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.258+0000 7f79ea085700 1 Processor -- start 2024-03-21T12:46:17.279 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.258+0000 7f79ea085700 1 -- start start 2024-03-21T12:46:17.279 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.258+0000 7f79ea085700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f79e4079fe0 con 0x7f79e4106a40 2024-03-21T12:46:17.279 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.258+0000 7f79e37fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f79e4106a40 0x7f79e4108e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41486/0 (socket says 172.21.15.43:41486) 2024-03-21T12:46:17.279 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.258+0000 7f79e37fe700 1 -- 172.21.15.43:0/3193946822 learned_addr learned my addr 172.21.15.43:0/3193946822 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:17.279 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.258+0000 7f79e27fc700 1 -- 172.21.15.43:0/3193946822 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1360503744 0 0) 0x7f79e4079fe0 con 0x7f79e4106a40 2024-03-21T12:46:17.279 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.259+0000 7f79e27fc700 1 -- 172.21.15.43:0/3193946822 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79cc0035b0 con 0x7f79e4106a40 2024-03-21T12:46:17.279 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.259+0000 7f79e27fc700 1 -- 172.21.15.43:0/3193946822 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2998256221 0 0) 0x7f79cc0035b0 con 0x7f79e4106a40 2024-03-21T12:46:17.279 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.259+0000 7f79e27fc700 1 -- 172.21.15.43:0/3193946822 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e410b560 con 0x7f79e4106a40 2024-03-21T12:46:17.279 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.259+0000 7f79e27fc700 1 -- 172.21.15.43:0/3193946822 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f79d8001580 con 0x7f79e4106a40 2024-03-21T12:46:17.280 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.259+0000 7f79e27fc700 1 -- 172.21.15.43:0/3193946822 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7f79d8001d20 con 0x7f79e4106a40 2024-03-21T12:46:17.280 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.259+0000 7f79e27fc700 1 -- 172.21.15.43:0/3193946822 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f79d8005960 con 0x7f79e4106a40 2024-03-21T12:46:17.280 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.259+0000 7f79ea085700 1 -- 172.21.15.43:0/3193946822 >> v1:172.21.15.43:6789/0 conn(0x7f79e4106a40 legacy=0x7f79e4108e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:17.280 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.259+0000 7f79ea085700 1 -- 172.21.15.43:0/3193946822 shutdown_connections 2024-03-21T12:46:17.280 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.259+0000 7f79ea085700 1 -- 172.21.15.43:0/3193946822 >> 172.21.15.43:0/3193946822 conn(0x7f79e4100510 msgr2=0x7f79e4102930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:17.280 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.259+0000 7f79ea085700 1 -- 172.21.15.43:0/3193946822 shutdown_connections 2024-03-21T12:46:17.280 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.260+0000 7f79ea085700 1 -- 172.21.15.43:0/3193946822 wait complete. 2024-03-21T12:46:17.280 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.260+0000 7f79ea085700 1 Processor -- start 2024-03-21T12:46:17.280 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.260+0000 7f79ea085700 1 -- start start 2024-03-21T12:46:17.280 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.260+0000 7f79ea085700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f79e419a220 con 0x7f79e4106a40 2024-03-21T12:46:17.280 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.260+0000 7f79e37fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f79e4106a40 0x7f79e4199b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41494/0 (socket says 172.21.15.43:41494) 2024-03-21T12:46:17.280 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.260+0000 7f79e37fe700 1 -- 172.21.15.43:0/3148790555 learned_addr learned my addr 172.21.15.43:0/3148790555 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:17.280 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.261+0000 7f79e0ff9700 1 -- 172.21.15.43:0/3148790555 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4146250131 0 0) 0x7f79e419a220 con 0x7f79e4106a40 2024-03-21T12:46:17.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.261+0000 7f79e0ff9700 1 -- 172.21.15.43:0/3148790555 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79cc0055f0 con 0x7f79e4106a40 2024-03-21T12:46:17.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.261+0000 7f79e0ff9700 1 -- 172.21.15.43:0/3148790555 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 685550263 0 0) 0x7f79cc0055f0 con 0x7f79e4106a40 2024-03-21T12:46:17.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.261+0000 7f79e0ff9700 1 -- 172.21.15.43:0/3148790555 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f79e419a220 con 0x7f79e4106a40 2024-03-21T12:46:17.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.261+0000 7f79e0ff9700 1 -- 172.21.15.43:0/3148790555 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f79d8004870 con 0x7f79e4106a40 2024-03-21T12:46:17.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.261+0000 7f79e0ff9700 1 -- 172.21.15.43:0/3148790555 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 969804153 0 0) 0x7f79e419a220 con 0x7f79e4106a40 2024-03-21T12:46:17.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.261+0000 7f79e0ff9700 1 -- 172.21.15.43:0/3148790555 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e419a3f0 con 0x7f79e4106a40 2024-03-21T12:46:17.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.261+0000 7f79ea085700 1 -- 172.21.15.43:0/3148790555 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e419a6e0 con 0x7f79e4106a40 2024-03-21T12:46:17.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.261+0000 7f79ea085700 1 -- 172.21.15.43:0/3148790555 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e419e1e0 con 0x7f79e4106a40 2024-03-21T12:46:17.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.261+0000 7f79e0ff9700 1 -- 172.21.15.43:0/3148790555 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7f79d8001ce0 con 0x7f79e4106a40 2024-03-21T12:46:17.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.261+0000 7f79e0ff9700 1 -- 172.21.15.43:0/3148790555 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f79d8006070 con 0x7f79e4106a40 2024-03-21T12:46:17.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.262+0000 7f79e0ff9700 1 -- 172.21.15.43:0/3148790555 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 4) v1 ==== 46248+0+0 (unknown 3190036632 0 0) 0x7f79d8011780 con 0x7f79e4106a40 2024-03-21T12:46:17.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.262+0000 7f79ea085700 1 -- 172.21.15.43:0/3148790555 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d0005150 con 0x7f79e4106a40 2024-03-21T12:46:17.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.262+0000 7f79e0ff9700 1 -- 172.21.15.43:0/3148790555 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 400852023 0 0) 0x7f79d8047a90 con 0x7f79e4106a40 2024-03-21T12:46:17.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.267+0000 7f79e0ff9700 1 -- 172.21.15.43:0/3148790555 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f79d8017630 con 0x7f79e4106a40 2024-03-21T12:46:17.281 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.529+0000 7f79ea085700 1 -- 172.21.15.43:0/3148790555 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f79d0005440 con 0x7f79e4106a40 2024-03-21T12:46:17.282 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.687+0000 7f79e0ff9700 1 -- 172.21.15.43:0/3148790555 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (unknown 2263024820 0 0) 0x7f79d8016f30 con 0x7f79e4106a40 2024-03-21T12:46:17.282 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.689+0000 7f79e0ff9700 1 -- 172.21.15.43:0/3148790555 <== mon.0 v1:172.21.15.43:6789/0 11 ==== mgrmap(e 5) v1 ==== 46259+0+0 (unknown 2743456346 0 0) 0x7f79d8046480 con 0x7f79e4106a40 2024-03-21T12:46:17.282 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.690+0000 7f79ea085700 1 -- 172.21.15.43:0/3148790555 >> v1:172.21.15.43:6800/2056453291 conn(0x7f79cc03a470 legacy=0x7f79cc03c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:17.282 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.690+0000 7f79ea085700 1 -- 172.21.15.43:0/3148790555 >> v1:172.21.15.43:6789/0 conn(0x7f79e4106a40 legacy=0x7f79e4199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:17.282 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.690+0000 7f79ea085700 1 -- 172.21.15.43:0/3148790555 shutdown_connections 2024-03-21T12:46:17.282 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.690+0000 7f79ea085700 1 -- 172.21.15.43:0/3148790555 >> 172.21.15.43:0/3148790555 conn(0x7f79e4100510 msgr2=0x7f79e4190810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:17.282 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.690+0000 7f79ea085700 1 -- 172.21.15.43:0/3148790555 shutdown_connections 2024-03-21T12:46:17.282 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:16.690+0000 7f79ea085700 1 -- 172.21.15.43:0/3148790555 wait complete. 2024-03-21T12:46:17.529 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:17 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:17.260+0000 7ffb3e595200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T12:46:17.529 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:17 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:17.360+0000 7ffb3e595200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T12:46:17.958 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:17 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3148790555' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-21T12:46:17.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:17 smithi043 ceph-mon[30062]: mgrmap e5: y(active, since 6s) 2024-03-21T12:46:18.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:18.033+0000 7ffb3e595200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T12:46:18.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:18.135+0000 7ffb3e595200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T12:46:18.661 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:18.279+0000 7ffb3e595200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T12:46:18.661 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:18.377+0000 7ffb3e595200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T12:46:18.661 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:18.463+0000 7ffb3e595200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T12:46:18.668 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-03-21T12:46:18.668 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-21T12:46:18.668 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-21T12:46:18.668 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-03-21T12:46:18.668 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-21T12:46:18.668 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-03-21T12:46:18.669 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.753+0000 7f89f7f87700 1 Processor -- start 2024-03-21T12:46:18.669 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.753+0000 7f89f7f87700 1 -- start start 2024-03-21T12:46:18.669 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.753+0000 7f89f7f87700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f89f010a410 con 0x7f89f0105fe0 2024-03-21T12:46:18.669 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.753+0000 7f89f5d23700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f89f0105fe0 0x7f89f0106400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41524/0 (socket says 172.21.15.43:41524) 2024-03-21T12:46:18.669 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.753+0000 7f89f5d23700 1 -- 172.21.15.43:0/3530458197 learned_addr learned my addr 172.21.15.43:0/3530458197 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:18.669 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.754+0000 7f89f4d21700 1 -- 172.21.15.43:0/3530458197 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2686123822 0 0) 0x7f89f010a410 con 0x7f89f0105fe0 2024-03-21T12:46:18.669 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.754+0000 7f89f4d21700 1 -- 172.21.15.43:0/3530458197 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89dc0035b0 con 0x7f89f0105fe0 2024-03-21T12:46:18.669 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.754+0000 7f89f4d21700 1 -- 172.21.15.43:0/3530458197 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2543273124 0 0) 0x7f89dc0035b0 con 0x7f89f0105fe0 2024-03-21T12:46:18.669 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.754+0000 7f89f4d21700 1 -- 172.21.15.43:0/3530458197 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f010b5f0 con 0x7f89f0105fe0 2024-03-21T12:46:18.669 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.754+0000 7f89f4d21700 1 -- 172.21.15.43:0/3530458197 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f89e0001580 con 0x7f89f0105fe0 2024-03-21T12:46:18.669 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.754+0000 7f89f4d21700 1 -- 172.21.15.43:0/3530458197 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7f89e0001d20 con 0x7f89f0105fe0 2024-03-21T12:46:18.669 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.754+0000 7f89f4d21700 1 -- 172.21.15.43:0/3530458197 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f89e0005960 con 0x7f89f0105fe0 2024-03-21T12:46:18.669 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.755+0000 7f89f7f87700 1 -- 172.21.15.43:0/3530458197 >> v1:172.21.15.43:6789/0 conn(0x7f89f0105fe0 legacy=0x7f89f0106400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:18.669 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.755+0000 7f89f7f87700 1 -- 172.21.15.43:0/3530458197 shutdown_connections 2024-03-21T12:46:18.670 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.755+0000 7f89f7f87700 1 -- 172.21.15.43:0/3530458197 >> 172.21.15.43:0/3530458197 conn(0x7f89f01015a0 msgr2=0x7f89f01039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:18.670 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.755+0000 7f89f7f87700 1 -- 172.21.15.43:0/3530458197 shutdown_connections 2024-03-21T12:46:18.670 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.755+0000 7f89f7f87700 1 -- 172.21.15.43:0/3530458197 wait complete. 2024-03-21T12:46:18.670 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.756+0000 7f89f7f87700 1 Processor -- start 2024-03-21T12:46:18.670 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.756+0000 7f89f7f87700 1 -- start start 2024-03-21T12:46:18.670 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.756+0000 7f89f7f87700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f89f019a1c0 con 0x7f89f0105fe0 2024-03-21T12:46:18.670 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.756+0000 7f89f5d23700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f89f0105fe0 0x7f89f0199ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41534/0 (socket says 172.21.15.43:41534) 2024-03-21T12:46:18.670 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.756+0000 7f89f5d23700 1 -- 172.21.15.43:0/752899043 learned_addr learned my addr 172.21.15.43:0/752899043 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:18.670 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.756+0000 7f89e6ffd700 1 -- 172.21.15.43:0/752899043 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1601335752 0 0) 0x7f89f019a1c0 con 0x7f89f0105fe0 2024-03-21T12:46:18.670 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.756+0000 7f89e6ffd700 1 -- 172.21.15.43:0/752899043 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89dc0055f0 con 0x7f89f0105fe0 2024-03-21T12:46:18.670 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.756+0000 7f89e6ffd700 1 -- 172.21.15.43:0/752899043 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3149546213 0 0) 0x7f89dc0055f0 con 0x7f89f0105fe0 2024-03-21T12:46:18.670 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.756+0000 7f89e6ffd700 1 -- 172.21.15.43:0/752899043 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f89f019a1c0 con 0x7f89f0105fe0 2024-03-21T12:46:18.670 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.756+0000 7f89e6ffd700 1 -- 172.21.15.43:0/752899043 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f89e0004870 con 0x7f89f0105fe0 2024-03-21T12:46:18.670 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.757+0000 7f89e6ffd700 1 -- 172.21.15.43:0/752899043 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2976565447 0 0) 0x7f89f019a1c0 con 0x7f89f0105fe0 2024-03-21T12:46:18.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.757+0000 7f89e6ffd700 1 -- 172.21.15.43:0/752899043 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f019a390 con 0x7f89f0105fe0 2024-03-21T12:46:18.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.757+0000 7f89f7f87700 1 -- 172.21.15.43:0/752899043 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f019a680 con 0x7f89f0105fe0 2024-03-21T12:46:18.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.757+0000 7f89f7f87700 1 -- 172.21.15.43:0/752899043 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f019e1d0 con 0x7f89f0105fe0 2024-03-21T12:46:18.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.757+0000 7f89e6ffd700 1 -- 172.21.15.43:0/752899043 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7f89e0001c00 con 0x7f89f0105fe0 2024-03-21T12:46:18.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.757+0000 7f89e6ffd700 1 -- 172.21.15.43:0/752899043 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f89e0005f20 con 0x7f89f0105fe0 2024-03-21T12:46:18.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.757+0000 7f89e6ffd700 1 -- 172.21.15.43:0/752899043 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 5) v1 ==== 46259+0+0 (unknown 2743456346 0 0) 0x7f89e0011580 con 0x7f89f0105fe0 2024-03-21T12:46:18.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.758+0000 7f89e6ffd700 1 -- 172.21.15.43:0/752899043 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 400852023 0 0) 0x7f89e0048720 con 0x7f89f0105fe0 2024-03-21T12:46:18.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.758+0000 7f89f5522700 1 -- 172.21.15.43:0/752899043 >> v1:172.21.15.43:6800/2056453291 conn(0x7f89dc03a510 legacy=0x7f89dc03c9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.43:6800/2056453291 2024-03-21T12:46:18.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.758+0000 7f89f7f87700 1 -- 172.21.15.43:0/752899043 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d4005150 con 0x7f89f0105fe0 2024-03-21T12:46:18.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.763+0000 7f89e6ffd700 1 -- 172.21.15.43:0/752899043 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f89e0017490 con 0x7f89f0105fe0 2024-03-21T12:46:18.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:17.959+0000 7f89f5522700 1 -- 172.21.15.43:0/752899043 >> v1:172.21.15.43:6800/2056453291 conn(0x7f89dc03a510 legacy=0x7f89dc03c9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.43:6800/2056453291 2024-03-21T12:46:18.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:18.030+0000 7f89f7f87700 1 -- 172.21.15.43:0/752899043 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f89d4005c50 con 0x7f89f0105fe0 2024-03-21T12:46:18.671 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:18.030+0000 7f89e6ffd700 1 -- 172.21.15.43:0/752899043 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (unknown 3768197548 0 15966916) 0x7f89e0016d90 con 0x7f89f0105fe0 2024-03-21T12:46:18.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:18.032+0000 7f89f7f87700 1 -- 172.21.15.43:0/752899043 >> v1:172.21.15.43:6800/2056453291 conn(0x7f89dc03a510 legacy=0x7f89dc03c9d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T12:46:18.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:18.033+0000 7f89f7f87700 1 -- 172.21.15.43:0/752899043 >> v1:172.21.15.43:6789/0 conn(0x7f89f0105fe0 legacy=0x7f89f0199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:18.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:18.033+0000 7f89f7f87700 1 -- 172.21.15.43:0/752899043 shutdown_connections 2024-03-21T12:46:18.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:18.033+0000 7f89f7f87700 1 -- 172.21.15.43:0/752899043 >> 172.21.15.43:0/752899043 conn(0x7f89f01015a0 msgr2=0x7f89f0103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:18.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:18.033+0000 7f89f7f87700 1 -- 172.21.15.43:0/752899043 shutdown_connections 2024-03-21T12:46:18.672 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:18.033+0000 7f89f7f87700 1 -- 172.21.15.43:0/752899043 wait complete. 2024-03-21T12:46:18.672 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for the mgr to restart... 2024-03-21T12:46:18.672 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr epoch 5... 2024-03-21T12:46:18.924 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:18 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/752899043' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-21T12:46:18.924 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:18.744+0000 7ffb3e595200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T12:46:18.924 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:18.857+0000 7ffb3e595200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T12:46:19.228 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:18.949+0000 7ffb3e595200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T12:46:19.229 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:19.047+0000 7ffb3e595200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T12:46:19.229 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:19.228+0000 7ffb3e595200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T12:46:20.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:20 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:20.215+0000 7ffb3e595200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T12:46:20.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:20 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:20.349+0000 7ffb3e595200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T12:46:20.979 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:20 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:20.687+0000 7ffb3e595200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T12:46:20.979 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:20 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:20.778+0000 7ffb3e595200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T12:46:20.979 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:20 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:20.979+0000 7ffb3e595200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T12:46:21.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:21 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:21.169+0000 7ffb3e595200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T12:46:21.720 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:21 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:21.461+0000 7ffb3e595200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T12:46:21.721 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:21 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:21.721+0000 7ffb3e595200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T12:46:21.983 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:21 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:21.963+0000 7ffb3e595200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T12:46:23.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:22 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:22.789+0000 7ffb3e595200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T12:46:23.689 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:23 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:23.368+0000 7ffb3e595200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T12:46:23.689 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:23 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:23.468+0000 7ffb3e595200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T12:46:23.690 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:23 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:23.689+0000 7ffb3e595200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T12:46:23.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:23 smithi043 ceph-mon[30062]: Active manager daemon y restarted 2024-03-21T12:46:23.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:23 smithi043 ceph-mon[30062]: Activating manager daemon y 2024-03-21T12:46:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:23 smithi043 ceph-mon[30062]: osdmap e2: 0 total, 0 up, 0 in 2024-03-21T12:46:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:23 smithi043 ceph-mon[30062]: mgrmap e6: y(active, starting, since 0.003015s) 2024-03-21T12:46:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:23 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T12:46:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:23 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T12:46:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:23 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T12:46:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:23 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T12:46:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:23 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T12:46:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:23 smithi043 ceph-mon[30062]: Manager daemon y is now available 2024-03-21T12:46:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:23 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' 2024-03-21T12:46:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:23 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' 2024-03-21T12:46:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:23 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:46:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:23 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:46:24.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:24 smithi043 ceph-mon[30062]: Found migration_current of "None". Setting to last migration. 2024-03-21T12:46:24.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:24 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-21T12:46:24.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:24 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-21T12:46:24.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:24 smithi043 ceph-mon[30062]: from='client.14124 v1:172.21.15.43:0/4122218377' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-21T12:46:24.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:24 smithi043 ceph-mon[30062]: mgrmap e7: y(active, since 1.00505s) 2024-03-21T12:46:24.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:24 smithi043 ceph-mon[30062]: from='client.14124 v1:172.21.15.43:0/4122218377' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-21T12:46:24.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:24 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' 2024-03-21T12:46:24.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:24 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' 2024-03-21T12:46:25.333 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-03-21T12:46:25.334 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-21T12:46:25.334 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-21T12:46:25.334 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-03-21T12:46:25.334 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.162+0000 7fe8e8626700 1 Processor -- start 2024-03-21T12:46:25.334 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.162+0000 7fe8e8626700 1 -- start start 2024-03-21T12:46:25.334 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.162+0000 7fe8e8626700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe8e0079f20 con 0x7fe8e0106b30 2024-03-21T12:46:25.334 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.162+0000 7fe8e63c2700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe8e0106b30 0x7fe8e0108f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41542/0 (socket says 172.21.15.43:41542) 2024-03-21T12:46:25.334 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.162+0000 7fe8e63c2700 1 -- 172.21.15.43:0/1386025491 learned_addr learned my addr 172.21.15.43:0/1386025491 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:25.334 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.162+0000 7fe8e53c0700 1 -- 172.21.15.43:0/1386025491 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1931732363 0 0) 0x7fe8e0079f20 con 0x7fe8e0106b30 2024-03-21T12:46:25.334 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.162+0000 7fe8e53c0700 1 -- 172.21.15.43:0/1386025491 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8cc0035b0 con 0x7fe8e0106b30 2024-03-21T12:46:25.334 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.163+0000 7fe8e53c0700 1 -- 172.21.15.43:0/1386025491 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 323564476 0 0) 0x7fe8cc0035b0 con 0x7fe8e0106b30 2024-03-21T12:46:25.334 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.163+0000 7fe8e53c0700 1 -- 172.21.15.43:0/1386025491 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e010b650 con 0x7fe8e0106b30 2024-03-21T12:46:25.334 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.163+0000 7fe8e53c0700 1 -- 172.21.15.43:0/1386025491 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fe8dc001580 con 0x7fe8e0106b30 2024-03-21T12:46:25.335 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.163+0000 7fe8e53c0700 1 -- 172.21.15.43:0/1386025491 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7fe8dc001d20 con 0x7fe8e0106b30 2024-03-21T12:46:25.335 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.163+0000 7fe8e53c0700 1 -- 172.21.15.43:0/1386025491 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fe8dc005960 con 0x7fe8e0106b30 2024-03-21T12:46:25.335 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.163+0000 7fe8e8626700 1 -- 172.21.15.43:0/1386025491 >> v1:172.21.15.43:6789/0 conn(0x7fe8e0106b30 legacy=0x7fe8e0108f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:25.335 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.164+0000 7fe8e8626700 1 -- 172.21.15.43:0/1386025491 shutdown_connections 2024-03-21T12:46:25.335 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.164+0000 7fe8e8626700 1 -- 172.21.15.43:0/1386025491 >> 172.21.15.43:0/1386025491 conn(0x7fe8e0100500 msgr2=0x7fe8e0102920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:25.335 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.164+0000 7fe8e8626700 1 -- 172.21.15.43:0/1386025491 shutdown_connections 2024-03-21T12:46:25.335 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.164+0000 7fe8e8626700 1 -- 172.21.15.43:0/1386025491 wait complete. 2024-03-21T12:46:25.335 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.164+0000 7fe8e8626700 1 Processor -- start 2024-03-21T12:46:25.335 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.165+0000 7fe8e8626700 1 -- start start 2024-03-21T12:46:25.336 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.165+0000 7fe8e8626700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe8e019a1d0 con 0x7fe8e0106b30 2024-03-21T12:46:25.336 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.165+0000 7fe8e63c2700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe8e0106b30 0x7fe8e0199ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41554/0 (socket says 172.21.15.43:41554) 2024-03-21T12:46:25.336 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.165+0000 7fe8e63c2700 1 -- 172.21.15.43:0/4122218377 learned_addr learned my addr 172.21.15.43:0/4122218377 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:25.336 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.165+0000 7fe8d77fe700 1 -- 172.21.15.43:0/4122218377 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2347114735 0 0) 0x7fe8e019a1d0 con 0x7fe8e0106b30 2024-03-21T12:46:25.336 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.165+0000 7fe8d77fe700 1 -- 172.21.15.43:0/4122218377 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8cc0055f0 con 0x7fe8e0106b30 2024-03-21T12:46:25.336 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.165+0000 7fe8d77fe700 1 -- 172.21.15.43:0/4122218377 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1038044219 0 0) 0x7fe8cc0055f0 con 0x7fe8e0106b30 2024-03-21T12:46:25.336 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.165+0000 7fe8d77fe700 1 -- 172.21.15.43:0/4122218377 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe8e019a1d0 con 0x7fe8e0106b30 2024-03-21T12:46:25.336 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.165+0000 7fe8d77fe700 1 -- 172.21.15.43:0/4122218377 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fe8dc001080 con 0x7fe8e0106b30 2024-03-21T12:46:25.336 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.166+0000 7fe8d77fe700 1 -- 172.21.15.43:0/4122218377 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1164609822 0 0) 0x7fe8e019a1d0 con 0x7fe8e0106b30 2024-03-21T12:46:25.336 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.166+0000 7fe8d77fe700 1 -- 172.21.15.43:0/4122218377 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e019a3a0 con 0x7fe8e0106b30 2024-03-21T12:46:25.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.166+0000 7fe8e8626700 1 -- 172.21.15.43:0/4122218377 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e019a690 con 0x7fe8e0106b30 2024-03-21T12:46:25.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.166+0000 7fe8e8626700 1 -- 172.21.15.43:0/4122218377 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e019e1e0 con 0x7fe8e0106b30 2024-03-21T12:46:25.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.166+0000 7fe8d77fe700 1 -- 172.21.15.43:0/4122218377 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7fe8dc001c80 con 0x7fe8e0106b30 2024-03-21T12:46:25.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.166+0000 7fe8d77fe700 1 -- 172.21.15.43:0/4122218377 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fe8dc005fa0 con 0x7fe8e0106b30 2024-03-21T12:46:25.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.167+0000 7fe8d77fe700 1 -- 172.21.15.43:0/4122218377 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 5) v1 ==== 46259+0+0 (unknown 2743456346 0 0) 0x7fe8dc011600 con 0x7fe8e0106b30 2024-03-21T12:46:25.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.167+0000 7fe8d77fe700 1 -- 172.21.15.43:0/4122218377 --> v1:172.21.15.43:6800/2056453291 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe8dc001080 con 0x7fe8cc03a6b0 2024-03-21T12:46:25.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.167+0000 7fe8d77fe700 1 -- 172.21.15.43:0/4122218377 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (unknown 400852023 0 0) 0x7fe8dc048760 con 0x7fe8e0106b30 2024-03-21T12:46:25.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.167+0000 7fe8e5bc1700 1 -- 172.21.15.43:0/4122218377 >> v1:172.21.15.43:6800/2056453291 conn(0x7fe8cc03a6b0 legacy=0x7fe8cc03cb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.43:6800/2056453291 2024-03-21T12:46:25.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.368+0000 7fe8e5bc1700 1 -- 172.21.15.43:0/4122218377 >> v1:172.21.15.43:6800/2056453291 conn(0x7fe8cc03a6b0 legacy=0x7fe8cc03cb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.43:6800/2056453291 2024-03-21T12:46:25.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:19.769+0000 7fe8e5bc1700 1 -- 172.21.15.43:0/4122218377 >> v1:172.21.15.43:6800/2056453291 conn(0x7fe8cc03a6b0 legacy=0x7fe8cc03cb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.43:6800/2056453291 2024-03-21T12:46:25.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:20.570+0000 7fe8e5bc1700 1 -- 172.21.15.43:0/4122218377 >> v1:172.21.15.43:6800/2056453291 conn(0x7fe8cc03a6b0 legacy=0x7fe8cc03cb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.43:6800/2056453291 2024-03-21T12:46:25.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:22.171+0000 7fe8e5bc1700 1 -- 172.21.15.43:0/4122218377 >> v1:172.21.15.43:6800/2056453291 conn(0x7fe8cc03a6b0 legacy=0x7fe8cc03cb70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.43:6800/2056453291 2024-03-21T12:46:25.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:23.705+0000 7fe8d77fe700 1 -- 172.21.15.43:0/4122218377 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mgrmap(e 6) v1 ==== 46061+0+0 (unknown 3974084755 0 0) 0x7fe8dc012b70 con 0x7fe8e0106b30 2024-03-21T12:46:25.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:23.705+0000 7fe8d77fe700 1 -- 172.21.15.43:0/4122218377 >> v1:172.21.15.43:6800/2056453291 conn(0x7fe8cc03a6b0 legacy=0x7fe8cc03cb70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T12:46:25.337 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:24.706+0000 7fe8d77fe700 1 -- 172.21.15.43:0/4122218377 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mgrmap(e 7) v1 ==== 46153+0+0 (unknown 4026471238 0 0) 0x7fe8dc049180 con 0x7fe8e0106b30 2024-03-21T12:46:25.338 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:24.706+0000 7fe8d77fe700 1 -- 172.21.15.43:0/4122218377 --> v1:172.21.15.43:6800/1388905519 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe8dc001080 con 0x7fe8cc03ef20 2024-03-21T12:46:25.338 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:24.707+0000 7fe8d77fe700 1 -- 172.21.15.43:0/4122218377 <== mgr.14120 v1:172.21.15.43:6800/1388905519 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (unknown 0 0 4109177037) 0x7fe8dc001080 con 0x7fe8cc03ef20 2024-03-21T12:46:25.338 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:24.714+0000 7fe8e8626700 1 -- 172.21.15.43:0/4122218377 --> v1:172.21.15.43:6800/1388905519 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fe8c4002630 con 0x7fe8cc03ef20 2024-03-21T12:46:25.338 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:24.714+0000 7fe8d77fe700 1 -- 172.21.15.43:0/4122218377 <== mgr.14120 v1:172.21.15.43:6800/1388905519 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (unknown 0 0 96372106) 0x7fe8c4002630 con 0x7fe8cc03ef20 2024-03-21T12:46:25.338 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:24.714+0000 7fe8e8626700 1 -- 172.21.15.43:0/4122218377 >> v1:172.21.15.43:6800/1388905519 conn(0x7fe8cc03ef20 legacy=0x7fe8cc041310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:25.338 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:24.714+0000 7fe8e8626700 1 -- 172.21.15.43:0/4122218377 >> v1:172.21.15.43:6789/0 conn(0x7fe8e0106b30 legacy=0x7fe8e0199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:25.338 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:24.714+0000 7fe8e8626700 1 -- 172.21.15.43:0/4122218377 shutdown_connections 2024-03-21T12:46:25.338 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:24.714+0000 7fe8e8626700 1 -- 172.21.15.43:0/4122218377 >> 172.21.15.43:0/4122218377 conn(0x7fe8e0100500 msgr2=0x7fe8e0190840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:25.338 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:24.714+0000 7fe8e8626700 1 -- 172.21.15.43:0/4122218377 shutdown_connections 2024-03-21T12:46:25.338 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:24.714+0000 7fe8e8626700 1 -- 172.21.15.43:0/4122218377 wait complete. 2024-03-21T12:46:25.338 INFO:teuthology.orchestra.run.smithi043.stdout:mgr epoch 5 is available 2024-03-21T12:46:25.338 INFO:teuthology.orchestra.run.smithi043.stdout:Setting orchestrator backend to cephadm... 2024-03-21T12:46:26.605 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.774+0000 7f91d633c700 1 Processor -- start 2024-03-21T12:46:26.605 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.774+0000 7f91d633c700 1 -- start start 2024-03-21T12:46:26.605 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.774+0000 7f91d633c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f91d0109320 con 0x7f91d0104e70 2024-03-21T12:46:26.605 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.774+0000 7f91cffff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f91d0104e70 0x7f91d0105290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48846/0 (socket says 172.21.15.43:48846) 2024-03-21T12:46:26.605 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.774+0000 7f91cffff700 1 -- 172.21.15.43:0/2013557840 learned_addr learned my addr 172.21.15.43:0/2013557840 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:26.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.775+0000 7f91ceffd700 1 -- 172.21.15.43:0/2013557840 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2793177391 0 0) 0x7f91d0109320 con 0x7f91d0104e70 2024-03-21T12:46:26.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.775+0000 7f91ceffd700 1 -- 172.21.15.43:0/2013557840 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91b80035b0 con 0x7f91d0104e70 2024-03-21T12:46:26.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.775+0000 7f91ceffd700 1 -- 172.21.15.43:0/2013557840 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2462993309 0 0) 0x7f91b80035b0 con 0x7f91d0104e70 2024-03-21T12:46:26.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.775+0000 7f91ceffd700 1 -- 172.21.15.43:0/2013557840 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d010a500 con 0x7f91d0104e70 2024-03-21T12:46:26.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.775+0000 7f91ceffd700 1 -- 172.21.15.43:0/2013557840 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f91c0001580 con 0x7f91d0104e70 2024-03-21T12:46:26.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.775+0000 7f91ceffd700 1 -- 172.21.15.43:0/2013557840 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7f91c0001d20 con 0x7f91d0104e70 2024-03-21T12:46:26.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.775+0000 7f91ceffd700 1 -- 172.21.15.43:0/2013557840 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f91c0005960 con 0x7f91d0104e70 2024-03-21T12:46:26.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.775+0000 7f91d633c700 1 -- 172.21.15.43:0/2013557840 >> v1:172.21.15.43:6789/0 conn(0x7f91d0104e70 legacy=0x7f91d0105290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:26.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.775+0000 7f91d633c700 1 -- 172.21.15.43:0/2013557840 shutdown_connections 2024-03-21T12:46:26.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.775+0000 7f91d633c700 1 -- 172.21.15.43:0/2013557840 >> 172.21.15.43:0/2013557840 conn(0x7f91d0100460 msgr2=0x7f91d01028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:26.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.776+0000 7f91d633c700 1 -- 172.21.15.43:0/2013557840 shutdown_connections 2024-03-21T12:46:26.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.776+0000 7f91d633c700 1 -- 172.21.15.43:0/2013557840 wait complete. 2024-03-21T12:46:26.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.776+0000 7f91d633c700 1 Processor -- start 2024-03-21T12:46:26.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.776+0000 7f91d633c700 1 -- start start 2024-03-21T12:46:26.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.776+0000 7f91d633c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f91d0079be0 con 0x7f91d0104e70 2024-03-21T12:46:26.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.776+0000 7f91cffff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f91d0104e70 0x7f91d019c770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48856/0 (socket says 172.21.15.43:48856) 2024-03-21T12:46:26.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.776+0000 7f91cffff700 1 -- 172.21.15.43:0/1923032833 learned_addr learned my addr 172.21.15.43:0/1923032833 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:26.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.777+0000 7f91cd7fa700 1 -- 172.21.15.43:0/1923032833 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3995694365 0 0) 0x7f91d0079be0 con 0x7f91d0104e70 2024-03-21T12:46:26.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.777+0000 7f91cd7fa700 1 -- 172.21.15.43:0/1923032833 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91b80055f0 con 0x7f91d0104e70 2024-03-21T12:46:26.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.777+0000 7f91cd7fa700 1 -- 172.21.15.43:0/1923032833 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1938328729 0 0) 0x7f91b80055f0 con 0x7f91d0104e70 2024-03-21T12:46:26.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.777+0000 7f91cd7fa700 1 -- 172.21.15.43:0/1923032833 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f91d0079be0 con 0x7f91d0104e70 2024-03-21T12:46:26.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.777+0000 7f91cd7fa700 1 -- 172.21.15.43:0/1923032833 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f91c0001080 con 0x7f91d0104e70 2024-03-21T12:46:26.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.777+0000 7f91cd7fa700 1 -- 172.21.15.43:0/1923032833 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2396175514 0 0) 0x7f91d0079be0 con 0x7f91d0104e70 2024-03-21T12:46:26.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.777+0000 7f91cd7fa700 1 -- 172.21.15.43:0/1923032833 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d0079db0 con 0x7f91d0104e70 2024-03-21T12:46:26.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.777+0000 7f91d633c700 1 -- 172.21.15.43:0/1923032833 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d019de90 con 0x7f91d0104e70 2024-03-21T12:46:26.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.777+0000 7f91d633c700 1 -- 172.21.15.43:0/1923032833 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d019e230 con 0x7f91d0104e70 2024-03-21T12:46:26.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.777+0000 7f91cd7fa700 1 -- 172.21.15.43:0/1923032833 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7f91c0001cc0 con 0x7f91d0104e70 2024-03-21T12:46:26.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.777+0000 7f91cd7fa700 1 -- 172.21.15.43:0/1923032833 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f91c0006010 con 0x7f91d0104e70 2024-03-21T12:46:26.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.778+0000 7f91cd7fa700 1 -- 172.21.15.43:0/1923032833 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 8) v1 ==== 46259+0+0 (unknown 94073229 0 0) 0x7f91c0011730 con 0x7f91d0104e70 2024-03-21T12:46:26.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.778+0000 7f91d633c700 1 -- 172.21.15.43:0/1923032833 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91bc005150 con 0x7f91d0104e70 2024-03-21T12:46:26.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.778+0000 7f91cd7fa700 1 -- 172.21.15.43:0/1923032833 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 73304150 0 0) 0x7f91c0047a70 con 0x7f91d0104e70 2024-03-21T12:46:26.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:25.783+0000 7f91cd7fa700 1 -- 172.21.15.43:0/1923032833 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f91c0017650 con 0x7f91d0104e70 2024-03-21T12:46:26.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:26.005+0000 7f91d633c700 1 -- 172.21.15.43:0/1923032833 --> v1:172.21.15.43:6800/1388905519 -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f91bc000bc0 con 0x7f91b803a660 2024-03-21T12:46:26.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:26.010+0000 7f91cd7fa700 1 -- 172.21.15.43:0/1923032833 <== mgr.14120 v1:172.21.15.43:6800/1388905519 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (unknown 0 0 0) 0x7f91bc000bc0 con 0x7f91b803a660 2024-03-21T12:46:26.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:26.012+0000 7f91d633c700 1 -- 172.21.15.43:0/1923032833 >> v1:172.21.15.43:6800/1388905519 conn(0x7f91b803a660 legacy=0x7f91b803cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:26.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:26.012+0000 7f91d633c700 1 -- 172.21.15.43:0/1923032833 >> v1:172.21.15.43:6789/0 conn(0x7f91d0104e70 legacy=0x7f91d019c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:26.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:26.012+0000 7f91d633c700 1 -- 172.21.15.43:0/1923032833 shutdown_connections 2024-03-21T12:46:26.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:26.012+0000 7f91d633c700 1 -- 172.21.15.43:0/1923032833 >> 172.21.15.43:0/1923032833 conn(0x7f91d0100460 msgr2=0x7f91d01028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:26.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:26.012+0000 7f91d633c700 1 -- 172.21.15.43:0/1923032833 shutdown_connections 2024-03-21T12:46:26.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:26.012+0000 7f91d633c700 1 -- 172.21.15.43:0/1923032833 wait complete. 2024-03-21T12:46:26.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:26 smithi043 ceph-mon[30062]: mgrmap e8: y(active, since 2s) 2024-03-21T12:46:26.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:26 smithi043 ceph-mon[30062]: [21/Mar/2024:12:46:25] ENGINE Bus STARTING 2024-03-21T12:46:26.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:26 smithi043 ceph-mon[30062]: from='client.14132 v1:172.21.15.43:0/1923032833' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:46:26.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:26 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' 2024-03-21T12:46:26.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:26 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:46:26.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:26 smithi043 ceph-mon[30062]: [21/Mar/2024:12:46:26] ENGINE Serving on http://172.21.15.43:8765 2024-03-21T12:46:26.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:26 smithi043 ceph-mon[30062]: [21/Mar/2024:12:46:26] ENGINE Serving on https://172.21.15.43:7150 2024-03-21T12:46:26.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:26 smithi043 ceph-mon[30062]: [21/Mar/2024:12:46:26] ENGINE Bus STARTED 2024-03-21T12:46:26.861 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:26 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:46:27.921 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:27 smithi043 ceph-mon[30062]: from='client.14134 v1:172.21.15.43:0/2170902511' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:46:27.929 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-21T12:46:27.929 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.100+0000 7fa47d771700 1 Processor -- start 2024-03-21T12:46:27.929 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.100+0000 7fa47d771700 1 -- start start 2024-03-21T12:46:27.929 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.100+0000 7fa47d771700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa47807ac10 con 0x7fa47807c010 2024-03-21T12:46:27.929 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.100+0000 7fa476ffd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa47807c010 0x7fa47807a470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48868/0 (socket says 172.21.15.43:48868) 2024-03-21T12:46:27.929 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.100+0000 7fa476ffd700 1 -- 172.21.15.43:0/2075894635 learned_addr learned my addr 172.21.15.43:0/2075894635 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:27.929 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.100+0000 7fa475ffb700 1 -- 172.21.15.43:0/2075894635 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3804605871 0 0) 0x7fa47807ac10 con 0x7fa47807c010 2024-03-21T12:46:27.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.100+0000 7fa475ffb700 1 -- 172.21.15.43:0/2075894635 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4600035b0 con 0x7fa47807c010 2024-03-21T12:46:27.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.100+0000 7fa475ffb700 1 -- 172.21.15.43:0/2075894635 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2558354807 0 0) 0x7fa4600035b0 con 0x7fa47807c010 2024-03-21T12:46:27.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.100+0000 7fa475ffb700 1 -- 172.21.15.43:0/2075894635 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47807ade0 con 0x7fa47807c010 2024-03-21T12:46:27.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.101+0000 7fa475ffb700 1 -- 172.21.15.43:0/2075894635 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fa468001580 con 0x7fa47807c010 2024-03-21T12:46:27.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.101+0000 7fa475ffb700 1 -- 172.21.15.43:0/2075894635 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7fa468001d20 con 0x7fa47807c010 2024-03-21T12:46:27.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.101+0000 7fa475ffb700 1 -- 172.21.15.43:0/2075894635 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fa468005960 con 0x7fa47807c010 2024-03-21T12:46:27.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.101+0000 7fa47d771700 1 -- 172.21.15.43:0/2075894635 >> v1:172.21.15.43:6789/0 conn(0x7fa47807c010 legacy=0x7fa47807a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:27.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.101+0000 7fa47d771700 1 -- 172.21.15.43:0/2075894635 shutdown_connections 2024-03-21T12:46:27.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.101+0000 7fa47d771700 1 -- 172.21.15.43:0/2075894635 >> 172.21.15.43:0/2075894635 conn(0x7fa4781015a0 msgr2=0x7fa4781039c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:27.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.101+0000 7fa47d771700 1 -- 172.21.15.43:0/2075894635 shutdown_connections 2024-03-21T12:46:27.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.101+0000 7fa47d771700 1 -- 172.21.15.43:0/2075894635 wait complete. 2024-03-21T12:46:27.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.102+0000 7fa47d771700 1 Processor -- start 2024-03-21T12:46:27.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.102+0000 7fa47d771700 1 -- start start 2024-03-21T12:46:27.930 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.102+0000 7fa47d771700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa47819c3f0 con 0x7fa47807c010 2024-03-21T12:46:27.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.102+0000 7fa476ffd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa47807c010 0x7fa47819bce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48872/0 (socket says 172.21.15.43:48872) 2024-03-21T12:46:27.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.102+0000 7fa476ffd700 1 -- 172.21.15.43:0/2170902511 learned_addr learned my addr 172.21.15.43:0/2170902511 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:27.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.102+0000 7fa45ffff700 1 -- 172.21.15.43:0/2170902511 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1456950111 0 0) 0x7fa47819c3f0 con 0x7fa47807c010 2024-03-21T12:46:27.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.102+0000 7fa45ffff700 1 -- 172.21.15.43:0/2170902511 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4600055f0 con 0x7fa47807c010 2024-03-21T12:46:27.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.103+0000 7fa45ffff700 1 -- 172.21.15.43:0/2170902511 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3823370011 0 0) 0x7fa4600055f0 con 0x7fa47807c010 2024-03-21T12:46:27.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.103+0000 7fa45ffff700 1 -- 172.21.15.43:0/2170902511 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa47819c3f0 con 0x7fa47807c010 2024-03-21T12:46:27.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.103+0000 7fa45ffff700 1 -- 172.21.15.43:0/2170902511 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fa468001080 con 0x7fa47807c010 2024-03-21T12:46:27.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.103+0000 7fa45ffff700 1 -- 172.21.15.43:0/2170902511 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3852326350 0 0) 0x7fa47819c3f0 con 0x7fa47807c010 2024-03-21T12:46:27.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.103+0000 7fa45ffff700 1 -- 172.21.15.43:0/2170902511 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa47819c5c0 con 0x7fa47807c010 2024-03-21T12:46:27.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.103+0000 7fa47d771700 1 -- 172.21.15.43:0/2170902511 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa47819c8b0 con 0x7fa47807c010 2024-03-21T12:46:27.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.103+0000 7fa47d771700 1 -- 172.21.15.43:0/2170902511 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4781a03f0 con 0x7fa47807c010 2024-03-21T12:46:27.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.103+0000 7fa45ffff700 1 -- 172.21.15.43:0/2170902511 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7fa468004ab0 con 0x7fa47807c010 2024-03-21T12:46:27.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.103+0000 7fa45ffff700 1 -- 172.21.15.43:0/2170902511 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fa4680061d0 con 0x7fa47807c010 2024-03-21T12:46:27.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.104+0000 7fa45ffff700 1 -- 172.21.15.43:0/2170902511 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 8) v1 ==== 46259+0+0 (unknown 94073229 0 0) 0x7fa468011830 con 0x7fa47807c010 2024-03-21T12:46:27.931 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.104+0000 7fa47d771700 1 -- 172.21.15.43:0/2170902511 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa464005150 con 0x7fa47807c010 2024-03-21T12:46:27.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.104+0000 7fa45ffff700 1 -- 172.21.15.43:0/2170902511 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 73304150 0 0) 0x7fa468047920 con 0x7fa47807c010 2024-03-21T12:46:27.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.109+0000 7fa45ffff700 1 -- 172.21.15.43:0/2170902511 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7fa468017510 con 0x7fa47807c010 2024-03-21T12:46:27.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.315+0000 7fa47d771700 1 -- 172.21.15.43:0/2170902511 --> v1:172.21.15.43:6800/1388905519 -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fa464000bc0 con 0x7fa46003a6b0 2024-03-21T12:46:27.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.315+0000 7fa45ffff700 1 -- 172.21.15.43:0/2170902511 <== mgr.14120 v1:172.21.15.43:6800/1388905519 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (unknown 0 0 2070689548) 0x7fa464000bc0 con 0x7fa46003a6b0 2024-03-21T12:46:27.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.317+0000 7fa47d771700 1 -- 172.21.15.43:0/2170902511 >> v1:172.21.15.43:6800/1388905519 conn(0x7fa46003a6b0 legacy=0x7fa46003cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:27.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.317+0000 7fa47d771700 1 -- 172.21.15.43:0/2170902511 >> v1:172.21.15.43:6789/0 conn(0x7fa47807c010 legacy=0x7fa47819bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:27.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.317+0000 7fa47d771700 1 -- 172.21.15.43:0/2170902511 shutdown_connections 2024-03-21T12:46:27.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.317+0000 7fa47d771700 1 -- 172.21.15.43:0/2170902511 >> 172.21.15.43:0/2170902511 conn(0x7fa4781015a0 msgr2=0x7fa47810a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:27.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.318+0000 7fa47d771700 1 -- 172.21.15.43:0/2170902511 shutdown_connections 2024-03-21T12:46:27.932 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:27.318+0000 7fa47d771700 1 -- 172.21.15.43:0/2170902511 wait complete. 2024-03-21T12:46:27.932 INFO:teuthology.orchestra.run.smithi043.stdout:Generating ssh key... 2024-03-21T12:46:28.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:28 smithi043 ceph-mon[30062]: from='client.14136 v1:172.21.15.43:0/1843803741' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:46:28.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:28 smithi043 ceph-mon[30062]: Generating ssh key... 2024-03-21T12:46:29.402 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:29 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: Generating public/private rsa key pair. 2024-03-21T12:46:29.402 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:29 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: Your identification has been saved in /tmp/tmpjae8axta/key. 2024-03-21T12:46:29.402 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:29 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: Your public key has been saved in /tmp/tmpjae8axta/key.pub. 2024-03-21T12:46:29.403 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:29 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: The key fingerprint is: 2024-03-21T12:46:29.403 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:29 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: SHA256:sGj/JROdJXbT+PJ+97CTb2w+IoohnXURwJ5Z3/g1jxo ceph-c06c8d1a-e780-11ee-95cd-87774f69a715 2024-03-21T12:46:29.403 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:29 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: The key's randomart image is: 2024-03-21T12:46:29.403 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:29 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: +---[RSA 3072]----+ 2024-03-21T12:46:29.403 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:29 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: | .... | 2024-03-21T12:46:29.403 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:29 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: | . ..o | 2024-03-21T12:46:29.403 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:29 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: | . . *.* + | 2024-03-21T12:46:29.403 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:29 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: | . o * =.= o.| 2024-03-21T12:46:29.403 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:29 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: | o . S.o.. o.+| 2024-03-21T12:46:29.403 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:29 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: | . .. o.. Eo...| 2024-03-21T12:46:29.403 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:29 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: | ..+o . ooo | 2024-03-21T12:46:29.403 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:29 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: | ..o+ ..oo+*| 2024-03-21T12:46:29.403 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:29 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: | .... . +BB| 2024-03-21T12:46:29.403 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:29 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: +----[SHA256]-----+ 2024-03-21T12:46:29.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.387+0000 7f8f1cd15700 1 Processor -- start 2024-03-21T12:46:29.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.387+0000 7f8f1cd15700 1 -- start start 2024-03-21T12:46:29.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.387+0000 7f8f1cd15700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8f18109330 con 0x7f8f18104e80 2024-03-21T12:46:29.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.387+0000 7f8f1659c700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8f18104e80 0x7f8f181052a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48876/0 (socket says 172.21.15.43:48876) 2024-03-21T12:46:29.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.387+0000 7f8f1659c700 1 -- 172.21.15.43:0/2221019588 learned_addr learned my addr 172.21.15.43:0/2221019588 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:29.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.387+0000 7f8f1559a700 1 -- 172.21.15.43:0/2221019588 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3396143655 0 0) 0x7f8f18109330 con 0x7f8f18104e80 2024-03-21T12:46:29.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.387+0000 7f8f1559a700 1 -- 172.21.15.43:0/2221019588 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f000035b0 con 0x7f8f18104e80 2024-03-21T12:46:29.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.388+0000 7f8f1559a700 1 -- 172.21.15.43:0/2221019588 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 337579688 0 0) 0x7f8f000035b0 con 0x7f8f18104e80 2024-03-21T12:46:29.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.388+0000 7f8f1559a700 1 -- 172.21.15.43:0/2221019588 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f1810a510 con 0x7f8f18104e80 2024-03-21T12:46:29.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.388+0000 7f8f1559a700 1 -- 172.21.15.43:0/2221019588 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f8f08001580 con 0x7f8f18104e80 2024-03-21T12:46:29.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.388+0000 7f8f1559a700 1 -- 172.21.15.43:0/2221019588 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7f8f08001d20 con 0x7f8f18104e80 2024-03-21T12:46:29.693 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.388+0000 7f8f1559a700 1 -- 172.21.15.43:0/2221019588 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f8f08005960 con 0x7f8f18104e80 2024-03-21T12:46:29.694 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.388+0000 7f8f1cd15700 1 -- 172.21.15.43:0/2221019588 >> v1:172.21.15.43:6789/0 conn(0x7f8f18104e80 legacy=0x7f8f181052a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:29.694 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.388+0000 7f8f1cd15700 1 -- 172.21.15.43:0/2221019588 shutdown_connections 2024-03-21T12:46:29.694 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.388+0000 7f8f1cd15700 1 -- 172.21.15.43:0/2221019588 >> 172.21.15.43:0/2221019588 conn(0x7f8f18100470 msgr2=0x7f8f181028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:29.694 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.389+0000 7f8f1cd15700 1 -- 172.21.15.43:0/2221019588 shutdown_connections 2024-03-21T12:46:29.694 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.389+0000 7f8f1cd15700 1 -- 172.21.15.43:0/2221019588 wait complete. 2024-03-21T12:46:29.694 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.389+0000 7f8f1cd15700 1 Processor -- start 2024-03-21T12:46:29.694 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.389+0000 7f8f1cd15700 1 -- start start 2024-03-21T12:46:29.694 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.389+0000 7f8f1cd15700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8f18199130 con 0x7f8f18104e80 2024-03-21T12:46:29.694 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.390+0000 7f8f1659c700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8f18104e80 0x7f8f18198a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48890/0 (socket says 172.21.15.43:48890) 2024-03-21T12:46:29.694 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.390+0000 7f8f1659c700 1 -- 172.21.15.43:0/1843803741 learned_addr learned my addr 172.21.15.43:0/1843803741 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:29.694 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.390+0000 7f8f077fe700 1 -- 172.21.15.43:0/1843803741 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3720525552 0 0) 0x7f8f18199130 con 0x7f8f18104e80 2024-03-21T12:46:29.694 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.390+0000 7f8f077fe700 1 -- 172.21.15.43:0/1843803741 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f000055f0 con 0x7f8f18104e80 2024-03-21T12:46:29.694 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.390+0000 7f8f077fe700 1 -- 172.21.15.43:0/1843803741 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 341768055 0 0) 0x7f8f000055f0 con 0x7f8f18104e80 2024-03-21T12:46:29.694 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.390+0000 7f8f077fe700 1 -- 172.21.15.43:0/1843803741 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8f18199130 con 0x7f8f18104e80 2024-03-21T12:46:29.695 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.390+0000 7f8f077fe700 1 -- 172.21.15.43:0/1843803741 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f8f08001080 con 0x7f8f18104e80 2024-03-21T12:46:29.695 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.390+0000 7f8f077fe700 1 -- 172.21.15.43:0/1843803741 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2348684594 0 0) 0x7f8f18199130 con 0x7f8f18104e80 2024-03-21T12:46:29.695 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.390+0000 7f8f077fe700 1 -- 172.21.15.43:0/1843803741 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f18199300 con 0x7f8f18104e80 2024-03-21T12:46:29.695 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.390+0000 7f8f1cd15700 1 -- 172.21.15.43:0/1843803741 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f181995f0 con 0x7f8f18104e80 2024-03-21T12:46:29.695 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.390+0000 7f8f1cd15700 1 -- 172.21.15.43:0/1843803741 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f1819d140 con 0x7f8f18104e80 2024-03-21T12:46:29.695 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.390+0000 7f8f077fe700 1 -- 172.21.15.43:0/1843803741 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7f8f08001ba0 con 0x7f8f18104e80 2024-03-21T12:46:29.695 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.390+0000 7f8f077fe700 1 -- 172.21.15.43:0/1843803741 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f8f08005eb0 con 0x7f8f18104e80 2024-03-21T12:46:29.695 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.391+0000 7f8f077fe700 1 -- 172.21.15.43:0/1843803741 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 8) v1 ==== 46259+0+0 (unknown 94073229 0 0) 0x7f8f08011550 con 0x7f8f18104e80 2024-03-21T12:46:29.695 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.391+0000 7f8f1cd15700 1 -- 172.21.15.43:0/1843803741 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef8005150 con 0x7f8f18104e80 2024-03-21T12:46:29.695 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.391+0000 7f8f077fe700 1 -- 172.21.15.43:0/1843803741 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 73304150 0 0) 0x7f8f08047990 con 0x7f8f18104e80 2024-03-21T12:46:29.695 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.396+0000 7f8f077fe700 1 -- 172.21.15.43:0/1843803741 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f8f080174a0 con 0x7f8f18104e80 2024-03-21T12:46:29.695 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:28.601+0000 7f8f1cd15700 1 -- 172.21.15.43:0/1843803741 --> v1:172.21.15.43:6800/1388905519 -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ef8000bc0 con 0x7f8f0003a6b0 2024-03-21T12:46:29.695 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:29.095+0000 7f8f077fe700 1 -- 172.21.15.43:0/1843803741 <== mgr.14120 v1:172.21.15.43:6800/1388905519 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (unknown 0 0 0) 0x7f8ef8000bc0 con 0x7f8f0003a6b0 2024-03-21T12:46:29.695 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:29.097+0000 7f8f1cd15700 1 -- 172.21.15.43:0/1843803741 >> v1:172.21.15.43:6800/1388905519 conn(0x7f8f0003a6b0 legacy=0x7f8f0003cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:29.696 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:29.097+0000 7f8f1cd15700 1 -- 172.21.15.43:0/1843803741 >> v1:172.21.15.43:6789/0 conn(0x7f8f18104e80 legacy=0x7f8f18198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:29.696 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:29.097+0000 7f8f1cd15700 1 -- 172.21.15.43:0/1843803741 shutdown_connections 2024-03-21T12:46:29.696 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:29.097+0000 7f8f1cd15700 1 -- 172.21.15.43:0/1843803741 >> 172.21.15.43:0/1843803741 conn(0x7f8f18100470 msgr2=0x7f8f181028b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:29.696 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:29.097+0000 7f8f1cd15700 1 -- 172.21.15.43:0/1843803741 shutdown_connections 2024-03-21T12:46:29.696 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:29.097+0000 7f8f1cd15700 1 -- 172.21.15.43:0/1843803741 wait complete. 2024-03-21T12:46:30.197 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:30 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' 2024-03-21T12:46:30.198 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:30 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' 2024-03-21T12:46:31.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFXsdcBQ76aLNx5bU9JoOV3qORr7R3smNyU0R24jPLQEMM26vALGGFzsB8oqNcVFuH4zlOu/DxuIRHmzGwfxq0BWIrCXpU+fUkkUD6n/GpK7z+zxRvl3r6vhqEMRaY4qM9UU0KhNpt5R9q/xarMk0b3D1q9j07ECjxzJxMtkNe0PA1Br920txh3qE64chMfPRgqx7X4/Za4vqrTbvysSNppLflsC1vpEEruPgTMEax1PHFuwuskgcq5ky/y4NpRLkI2Zxy9qxr8IJrQqHja0Cy/hQkM2frnb3eUli3DOrvon2jr6A7HiIv6TG3q96pHFSMDLEi1plIkrOTlN67sB5aoG7rVDeNb3g/fBVuE8srqafOGlN9/UDEDAYZnLtOPR+APa4oihpNN3XOlu0Apq05gwkhUKHyU2BcXLJx4zShPMbXuOgU1y606A7qd+Er+kMW6GGTCuZkCvYDABnkmmuhjL4MEPHW4t4bGefm/ltN+LWTQ11AnSiSPVMe6LIkQBs= ceph-c06c8d1a-e780-11ee-95cd-87774f69a715 2024-03-21T12:46:31.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.198+0000 7f7fa7da3700 1 Processor -- start 2024-03-21T12:46:31.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.198+0000 7f7fa7da3700 1 -- start start 2024-03-21T12:46:31.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.198+0000 7f7fa7da3700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7fa0109330 con 0x7f7fa0104f10 2024-03-21T12:46:31.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.198+0000 7f7fa5b3f700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7fa0104f10 0x7f7fa0105330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48898/0 (socket says 172.21.15.43:48898) 2024-03-21T12:46:31.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.198+0000 7f7fa5b3f700 1 -- 172.21.15.43:0/175340843 learned_addr learned my addr 172.21.15.43:0/175340843 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:31.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.199+0000 7f7fa4b3d700 1 -- 172.21.15.43:0/175340843 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3811793845 0 0) 0x7f7fa0109330 con 0x7f7fa0104f10 2024-03-21T12:46:31.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.199+0000 7f7fa4b3d700 1 -- 172.21.15.43:0/175340843 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f8c0035b0 con 0x7f7fa0104f10 2024-03-21T12:46:31.016 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.199+0000 7f7fa4b3d700 1 -- 172.21.15.43:0/175340843 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 676111813 0 0) 0x7f7f8c0035b0 con 0x7f7fa0104f10 2024-03-21T12:46:31.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.199+0000 7f7fa4b3d700 1 -- 172.21.15.43:0/175340843 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa010a510 con 0x7f7fa0104f10 2024-03-21T12:46:31.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.199+0000 7f7fa4b3d700 1 -- 172.21.15.43:0/175340843 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f7f90001580 con 0x7f7fa0104f10 2024-03-21T12:46:31.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.199+0000 7f7fa4b3d700 1 -- 172.21.15.43:0/175340843 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7f7f90001d20 con 0x7f7fa0104f10 2024-03-21T12:46:31.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.199+0000 7f7fa4b3d700 1 -- 172.21.15.43:0/175340843 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f7f90005960 con 0x7f7fa0104f10 2024-03-21T12:46:31.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.199+0000 7f7fa7da3700 1 -- 172.21.15.43:0/175340843 >> v1:172.21.15.43:6789/0 conn(0x7f7fa0104f10 legacy=0x7f7fa0105330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:31.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.200+0000 7f7fa7da3700 1 -- 172.21.15.43:0/175340843 shutdown_connections 2024-03-21T12:46:31.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.200+0000 7f7fa7da3700 1 -- 172.21.15.43:0/175340843 >> 172.21.15.43:0/175340843 conn(0x7f7fa01004d0 msgr2=0x7f7fa01028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:31.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.200+0000 7f7fa7da3700 1 -- 172.21.15.43:0/175340843 shutdown_connections 2024-03-21T12:46:31.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.200+0000 7f7fa7da3700 1 -- 172.21.15.43:0/175340843 wait complete. 2024-03-21T12:46:31.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.200+0000 7f7fa7da3700 1 Processor -- start 2024-03-21T12:46:31.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.200+0000 7f7fa7da3700 1 -- start start 2024-03-21T12:46:31.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.200+0000 7f7fa7da3700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7fa019a5e0 con 0x7f7fa0104f10 2024-03-21T12:46:31.017 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.200+0000 7f7fa5b3f700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7fa0104f10 0x7f7fa019ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48906/0 (socket says 172.21.15.43:48906) 2024-03-21T12:46:31.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.201+0000 7f7fa5b3f700 1 -- 172.21.15.43:0/1316215322 learned_addr learned my addr 172.21.15.43:0/1316215322 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:31.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.201+0000 7f7f96ffd700 1 -- 172.21.15.43:0/1316215322 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2408279150 0 0) 0x7f7fa019a5e0 con 0x7f7fa0104f10 2024-03-21T12:46:31.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.201+0000 7f7f96ffd700 1 -- 172.21.15.43:0/1316215322 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f8c0055f0 con 0x7f7fa0104f10 2024-03-21T12:46:31.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.201+0000 7f7f96ffd700 1 -- 172.21.15.43:0/1316215322 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2126862856 0 0) 0x7f7f8c0055f0 con 0x7f7fa0104f10 2024-03-21T12:46:31.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.201+0000 7f7f96ffd700 1 -- 172.21.15.43:0/1316215322 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7fa019a5e0 con 0x7f7fa0104f10 2024-03-21T12:46:31.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.201+0000 7f7f96ffd700 1 -- 172.21.15.43:0/1316215322 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f7f90004870 con 0x7f7fa0104f10 2024-03-21T12:46:31.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.201+0000 7f7f96ffd700 1 -- 172.21.15.43:0/1316215322 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1414737226 0 0) 0x7f7fa019a5e0 con 0x7f7fa0104f10 2024-03-21T12:46:31.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.201+0000 7f7f96ffd700 1 -- 172.21.15.43:0/1316215322 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa0079b80 con 0x7f7fa0104f10 2024-03-21T12:46:31.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.201+0000 7f7fa7da3700 1 -- 172.21.15.43:0/1316215322 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa0079e90 con 0x7f7fa0104f10 2024-03-21T12:46:31.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.201+0000 7f7fa7da3700 1 -- 172.21.15.43:0/1316215322 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa01a04e0 con 0x7f7fa0104f10 2024-03-21T12:46:31.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.201+0000 7f7f96ffd700 1 -- 172.21.15.43:0/1316215322 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7f7f90001190 con 0x7f7fa0104f10 2024-03-21T12:46:31.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.201+0000 7f7f96ffd700 1 -- 172.21.15.43:0/1316215322 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f7f90005f70 con 0x7f7fa0104f10 2024-03-21T12:46:31.018 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.202+0000 7f7f96ffd700 1 -- 172.21.15.43:0/1316215322 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 8) v1 ==== 46259+0+0 (unknown 94073229 0 0) 0x7f7f900115d0 con 0x7f7fa0104f10 2024-03-21T12:46:31.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.202+0000 7f7fa7da3700 1 -- 172.21.15.43:0/1316215322 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f84005150 con 0x7f7fa0104f10 2024-03-21T12:46:31.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.202+0000 7f7f96ffd700 1 -- 172.21.15.43:0/1316215322 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 73304150 0 0) 0x7f7f90047960 con 0x7f7fa0104f10 2024-03-21T12:46:31.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.207+0000 7f7f96ffd700 1 -- 172.21.15.43:0/1316215322 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f7f900174e0 con 0x7f7fa0104f10 2024-03-21T12:46:31.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.413+0000 7f7fa7da3700 1 -- 172.21.15.43:0/1316215322 --> v1:172.21.15.43:6800/1388905519 -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f84000bc0 con 0x7f7f8c03a560 2024-03-21T12:46:31.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.414+0000 7f7f96ffd700 1 -- 172.21.15.43:0/1316215322 <== mgr.14120 v1:172.21.15.43:6800/1388905519 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (unknown 0 0 1805687953) 0x7f7f84000bc0 con 0x7f7f8c03a560 2024-03-21T12:46:31.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.416+0000 7f7fa7da3700 1 -- 172.21.15.43:0/1316215322 >> v1:172.21.15.43:6800/1388905519 conn(0x7f7f8c03a560 legacy=0x7f7f8c03ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:31.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.416+0000 7f7fa7da3700 1 -- 172.21.15.43:0/1316215322 >> v1:172.21.15.43:6789/0 conn(0x7f7fa0104f10 legacy=0x7f7fa019ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:31.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.416+0000 7f7fa7da3700 1 -- 172.21.15.43:0/1316215322 shutdown_connections 2024-03-21T12:46:31.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.416+0000 7f7fa7da3700 1 -- 172.21.15.43:0/1316215322 >> 172.21.15.43:0/1316215322 conn(0x7f7fa01004d0 msgr2=0x7f7fa01028c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:31.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.416+0000 7f7fa7da3700 1 -- 172.21.15.43:0/1316215322 shutdown_connections 2024-03-21T12:46:31.019 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:30.416+0000 7f7fa7da3700 1 -- 172.21.15.43:0/1316215322 wait complete. 2024-03-21T12:46:31.019 INFO:teuthology.orchestra.run.smithi043.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-21T12:46:31.019 INFO:teuthology.orchestra.run.smithi043.stdout:Adding key to root@localhost authorized_keys... 2024-03-21T12:46:31.019 INFO:teuthology.orchestra.run.smithi043.stdout:Adding host smithi043... 2024-03-21T12:46:31.345 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:31 smithi043 ceph-mon[30062]: from='client.14138 v1:172.21.15.43:0/1316215322' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:46:32.358 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:32 smithi043 ceph-mon[30062]: from='client.14140 v1:172.21.15.43:0/2750298690' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi043", "addr": "172.21.15.43", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:46:33.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:33 smithi043 ceph-mon[30062]: Deploying cephadm binary to smithi043 2024-03-21T12:46:36.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Added host 'smithi043' with addr '172.21.15.43' 2024-03-21T12:46:36.605 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.451+0000 7f2524abd700 1 Processor -- start 2024-03-21T12:46:36.605 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.451+0000 7f2524abd700 1 -- start start 2024-03-21T12:46:36.605 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.452+0000 7f2524abd700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f252007abf0 con 0x7f252007bff0 2024-03-21T12:46:36.605 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.452+0000 7f251e59c700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f252007bff0 0x7f252007a450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48912/0 (socket says 172.21.15.43:48912) 2024-03-21T12:46:36.605 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.452+0000 7f251e59c700 1 -- 172.21.15.43:0/140031606 learned_addr learned my addr 172.21.15.43:0/140031606 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:36.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.452+0000 7f251d59a700 1 -- 172.21.15.43:0/140031606 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3154681936 0 0) 0x7f252007abf0 con 0x7f252007bff0 2024-03-21T12:46:36.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.452+0000 7f251d59a700 1 -- 172.21.15.43:0/140031606 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25080035b0 con 0x7f252007bff0 2024-03-21T12:46:36.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.452+0000 7f251d59a700 1 -- 172.21.15.43:0/140031606 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1790823337 0 0) 0x7f25080035b0 con 0x7f252007bff0 2024-03-21T12:46:36.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.452+0000 7f251d59a700 1 -- 172.21.15.43:0/140031606 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252007adc0 con 0x7f252007bff0 2024-03-21T12:46:36.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.452+0000 7f251d59a700 1 -- 172.21.15.43:0/140031606 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f2510001580 con 0x7f252007bff0 2024-03-21T12:46:36.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.452+0000 7f251d59a700 1 -- 172.21.15.43:0/140031606 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7f2510001d20 con 0x7f252007bff0 2024-03-21T12:46:36.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.452+0000 7f251d59a700 1 -- 172.21.15.43:0/140031606 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f2510005960 con 0x7f252007bff0 2024-03-21T12:46:36.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.453+0000 7f2524abd700 1 -- 172.21.15.43:0/140031606 >> v1:172.21.15.43:6789/0 conn(0x7f252007bff0 legacy=0x7f252007a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:36.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.453+0000 7f2524abd700 1 -- 172.21.15.43:0/140031606 shutdown_connections 2024-03-21T12:46:36.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.453+0000 7f2524abd700 1 -- 172.21.15.43:0/140031606 >> 172.21.15.43:0/140031606 conn(0x7f2520101650 msgr2=0x7f2520103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:36.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.453+0000 7f2524abd700 1 -- 172.21.15.43:0/140031606 shutdown_connections 2024-03-21T12:46:36.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.453+0000 7f2524abd700 1 -- 172.21.15.43:0/140031606 wait complete. 2024-03-21T12:46:36.606 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.454+0000 7f2524abd700 1 Processor -- start 2024-03-21T12:46:36.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.454+0000 7f2524abd700 1 -- start start 2024-03-21T12:46:36.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.454+0000 7f2524abd700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f252019c440 con 0x7f252007bff0 2024-03-21T12:46:36.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.454+0000 7f251e59c700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f252007bff0 0x7f252019bd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48928/0 (socket says 172.21.15.43:48928) 2024-03-21T12:46:36.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.454+0000 7f251e59c700 1 -- 172.21.15.43:0/2750298690 learned_addr learned my addr 172.21.15.43:0/2750298690 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:36.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.454+0000 7f250f7fe700 1 -- 172.21.15.43:0/2750298690 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3988708000 0 0) 0x7f252019c440 con 0x7f252007bff0 2024-03-21T12:46:36.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.454+0000 7f250f7fe700 1 -- 172.21.15.43:0/2750298690 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25080055f0 con 0x7f252007bff0 2024-03-21T12:46:36.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.454+0000 7f250f7fe700 1 -- 172.21.15.43:0/2750298690 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1064614828 0 0) 0x7f25080055f0 con 0x7f252007bff0 2024-03-21T12:46:36.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.454+0000 7f250f7fe700 1 -- 172.21.15.43:0/2750298690 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f252019c440 con 0x7f252007bff0 2024-03-21T12:46:36.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.454+0000 7f250f7fe700 1 -- 172.21.15.43:0/2750298690 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f2510001080 con 0x7f252007bff0 2024-03-21T12:46:36.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.455+0000 7f250f7fe700 1 -- 172.21.15.43:0/2750298690 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2578029236 0 0) 0x7f252019c440 con 0x7f252007bff0 2024-03-21T12:46:36.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.455+0000 7f250f7fe700 1 -- 172.21.15.43:0/2750298690 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252019c610 con 0x7f252007bff0 2024-03-21T12:46:36.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.455+0000 7f2524abd700 1 -- 172.21.15.43:0/2750298690 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f252019c900 con 0x7f252007bff0 2024-03-21T12:46:36.607 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.455+0000 7f2524abd700 1 -- 172.21.15.43:0/2750298690 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f25201a0450 con 0x7f252007bff0 2024-03-21T12:46:36.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.455+0000 7f250f7fe700 1 -- 172.21.15.43:0/2750298690 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7f2510001ba0 con 0x7f252007bff0 2024-03-21T12:46:36.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.455+0000 7f250f7fe700 1 -- 172.21.15.43:0/2750298690 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f2510005ec0 con 0x7f252007bff0 2024-03-21T12:46:36.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.455+0000 7f250f7fe700 1 -- 172.21.15.43:0/2750298690 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 8) v1 ==== 46259+0+0 (unknown 94073229 0 0) 0x7f2510011540 con 0x7f252007bff0 2024-03-21T12:46:36.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.456+0000 7f2524abd700 1 -- 172.21.15.43:0/2750298690 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2500005150 con 0x7f252007bff0 2024-03-21T12:46:36.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.456+0000 7f250f7fe700 1 -- 172.21.15.43:0/2750298690 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 73304150 0 0) 0x7f2510047920 con 0x7f252007bff0 2024-03-21T12:46:36.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.460+0000 7f250f7fe700 1 -- 172.21.15.43:0/2750298690 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f2510017490 con 0x7f252007bff0 2024-03-21T12:46:36.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:31.670+0000 7f2524abd700 1 -- 172.21.15.43:0/2750298690 --> v1:172.21.15.43:6800/1388905519 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi043", "addr": "172.21.15.43", "target": ["mon-mgr", ""]}) v1 -- 0x7f2500000bc0 con 0x7f250803a6b0 2024-03-21T12:46:36.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:36.005+0000 7f250f7fe700 1 -- 172.21.15.43:0/2750298690 <== mgr.14120 v1:172.21.15.43:6800/1388905519 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (unknown 0 0 299949967) 0x7f2500000bc0 con 0x7f250803a6b0 2024-03-21T12:46:36.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:36.007+0000 7f2524abd700 1 -- 172.21.15.43:0/2750298690 >> v1:172.21.15.43:6800/1388905519 conn(0x7f250803a6b0 legacy=0x7f250803cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:36.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:36.007+0000 7f2524abd700 1 -- 172.21.15.43:0/2750298690 >> v1:172.21.15.43:6789/0 conn(0x7f252007bff0 legacy=0x7f252019bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:36.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:36.007+0000 7f2524abd700 1 -- 172.21.15.43:0/2750298690 shutdown_connections 2024-03-21T12:46:36.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:36.007+0000 7f2524abd700 1 -- 172.21.15.43:0/2750298690 >> 172.21.15.43:0/2750298690 conn(0x7f2520101650 msgr2=0x7f252010a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:36.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:36.007+0000 7f2524abd700 1 -- 172.21.15.43:0/2750298690 shutdown_connections 2024-03-21T12:46:36.608 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:36.007+0000 7f2524abd700 1 -- 172.21.15.43:0/2750298690 wait complete. 2024-03-21T12:46:36.608 INFO:teuthology.orchestra.run.smithi043.stdout:Deploying unmanaged mon service... 2024-03-21T12:46:37.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:37 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' 2024-03-21T12:46:37.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:37 smithi043 ceph-mon[30062]: Added host smithi043 2024-03-21T12:46:37.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:37 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:46:37.824 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-21T12:46:37.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.018+0000 7fb21fb4d700 1 Processor -- start 2024-03-21T12:46:37.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.018+0000 7fb21fb4d700 1 -- start start 2024-03-21T12:46:37.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.018+0000 7fb21fb4d700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb2180796f0 con 0x7fb218106aa0 2024-03-21T12:46:37.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.019+0000 7fb21d8e9700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb218106aa0 0x7fb218108e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52722/0 (socket says 172.21.15.43:52722) 2024-03-21T12:46:37.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.019+0000 7fb21d8e9700 1 -- 172.21.15.43:0/690450673 learned_addr learned my addr 172.21.15.43:0/690450673 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:37.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.019+0000 7fb21c8e7700 1 -- 172.21.15.43:0/690450673 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4198528193 0 0) 0x7fb2180796f0 con 0x7fb218106aa0 2024-03-21T12:46:37.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.019+0000 7fb21c8e7700 1 -- 172.21.15.43:0/690450673 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2000035b0 con 0x7fb218106aa0 2024-03-21T12:46:37.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.019+0000 7fb21c8e7700 1 -- 172.21.15.43:0/690450673 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 697131185 0 0) 0x7fb2000035b0 con 0x7fb218106aa0 2024-03-21T12:46:37.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.019+0000 7fb21c8e7700 1 -- 172.21.15.43:0/690450673 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21810b5c0 con 0x7fb218106aa0 2024-03-21T12:46:37.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.019+0000 7fb21c8e7700 1 -- 172.21.15.43:0/690450673 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fb210001900 con 0x7fb218106aa0 2024-03-21T12:46:37.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.019+0000 7fb21c8e7700 1 -- 172.21.15.43:0/690450673 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7fb2100020a0 con 0x7fb218106aa0 2024-03-21T12:46:37.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.019+0000 7fb21c8e7700 1 -- 172.21.15.43:0/690450673 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fb210005ce0 con 0x7fb218106aa0 2024-03-21T12:46:37.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.020+0000 7fb21fb4d700 1 -- 172.21.15.43:0/690450673 >> v1:172.21.15.43:6789/0 conn(0x7fb218106aa0 legacy=0x7fb218108e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:37.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.020+0000 7fb21fb4d700 1 -- 172.21.15.43:0/690450673 shutdown_connections 2024-03-21T12:46:37.825 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.020+0000 7fb21fb4d700 1 -- 172.21.15.43:0/690450673 >> 172.21.15.43:0/690450673 conn(0x7fb2181004f0 msgr2=0x7fb218102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:37.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.020+0000 7fb21fb4d700 1 -- 172.21.15.43:0/690450673 shutdown_connections 2024-03-21T12:46:37.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.020+0000 7fb21fb4d700 1 -- 172.21.15.43:0/690450673 wait complete. 2024-03-21T12:46:37.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.020+0000 7fb21fb4d700 1 Processor -- start 2024-03-21T12:46:37.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.021+0000 7fb21fb4d700 1 -- start start 2024-03-21T12:46:37.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.021+0000 7fb21fb4d700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb21819a160 con 0x7fb218106aa0 2024-03-21T12:46:37.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.021+0000 7fb21d8e9700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb218106aa0 0x7fb218199a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52732/0 (socket says 172.21.15.43:52732) 2024-03-21T12:46:37.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.021+0000 7fb21d8e9700 1 -- 172.21.15.43:0/353994849 learned_addr learned my addr 172.21.15.43:0/353994849 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:37.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.021+0000 7fb216ffd700 1 -- 172.21.15.43:0/353994849 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 461925725 0 0) 0x7fb21819a160 con 0x7fb218106aa0 2024-03-21T12:46:37.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.021+0000 7fb216ffd700 1 -- 172.21.15.43:0/353994849 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2000055f0 con 0x7fb218106aa0 2024-03-21T12:46:37.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.021+0000 7fb216ffd700 1 -- 172.21.15.43:0/353994849 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3754651701 0 0) 0x7fb2000055f0 con 0x7fb218106aa0 2024-03-21T12:46:37.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.021+0000 7fb216ffd700 1 -- 172.21.15.43:0/353994849 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb21819a160 con 0x7fb218106aa0 2024-03-21T12:46:37.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.021+0000 7fb216ffd700 1 -- 172.21.15.43:0/353994849 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fb210004bf0 con 0x7fb218106aa0 2024-03-21T12:46:37.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.021+0000 7fb216ffd700 1 -- 172.21.15.43:0/353994849 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3409408959 0 0) 0x7fb21819a160 con 0x7fb218106aa0 2024-03-21T12:46:37.826 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.021+0000 7fb216ffd700 1 -- 172.21.15.43:0/353994849 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb21819a330 con 0x7fb218106aa0 2024-03-21T12:46:37.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.021+0000 7fb21fb4d700 1 -- 172.21.15.43:0/353994849 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb21819a620 con 0x7fb218106aa0 2024-03-21T12:46:37.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.021+0000 7fb21fb4d700 1 -- 172.21.15.43:0/353994849 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb21819e170 con 0x7fb218106aa0 2024-03-21T12:46:37.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.021+0000 7fb216ffd700 1 -- 172.21.15.43:0/353994849 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7fb210001510 con 0x7fb218106aa0 2024-03-21T12:46:37.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.021+0000 7fb216ffd700 1 -- 172.21.15.43:0/353994849 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fb2100062f0 con 0x7fb218106aa0 2024-03-21T12:46:37.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.022+0000 7fb216ffd700 1 -- 172.21.15.43:0/353994849 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 8) v1 ==== 46259+0+0 (unknown 94073229 0 0) 0x7fb210011950 con 0x7fb218106aa0 2024-03-21T12:46:37.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.022+0000 7fb21fb4d700 1 -- 172.21.15.43:0/353994849 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb204005150 con 0x7fb218106aa0 2024-03-21T12:46:37.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.022+0000 7fb216ffd700 1 -- 172.21.15.43:0/353994849 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 73304150 0 0) 0x7fb210047d10 con 0x7fb218106aa0 2024-03-21T12:46:37.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.027+0000 7fb216ffd700 1 -- 172.21.15.43:0/353994849 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7fb210017860 con 0x7fb218106aa0 2024-03-21T12:46:37.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.249+0000 7fb21fb4d700 1 -- 172.21.15.43:0/353994849 --> v1:172.21.15.43:6800/1388905519 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fb204000bc0 con 0x7fb20003a560 2024-03-21T12:46:37.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.254+0000 7fb216ffd700 1 -- 172.21.15.43:0/353994849 <== mgr.14120 v1:172.21.15.43:6800/1388905519 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 3265049985) 0x7fb204000bc0 con 0x7fb20003a560 2024-03-21T12:46:37.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.256+0000 7fb21fb4d700 1 -- 172.21.15.43:0/353994849 >> v1:172.21.15.43:6800/1388905519 conn(0x7fb20003a560 legacy=0x7fb20003ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:37.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.256+0000 7fb21fb4d700 1 -- 172.21.15.43:0/353994849 >> v1:172.21.15.43:6789/0 conn(0x7fb218106aa0 legacy=0x7fb218199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:37.827 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.256+0000 7fb21fb4d700 1 -- 172.21.15.43:0/353994849 shutdown_connections 2024-03-21T12:46:37.828 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.256+0000 7fb21fb4d700 1 -- 172.21.15.43:0/353994849 >> 172.21.15.43:0/353994849 conn(0x7fb2181004f0 msgr2=0x7fb218190810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:37.828 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.256+0000 7fb21fb4d700 1 -- 172.21.15.43:0/353994849 shutdown_connections 2024-03-21T12:46:37.828 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:37.256+0000 7fb21fb4d700 1 -- 172.21.15.43:0/353994849 wait complete. 2024-03-21T12:46:37.828 INFO:teuthology.orchestra.run.smithi043.stdout:Deploying unmanaged mgr service... 2024-03-21T12:46:38.438 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:38 smithi043 ceph-mon[30062]: from='client.14142 v1:172.21.15.43:0/353994849' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:46:38.438 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:38 smithi043 ceph-mon[30062]: Saving service mon spec with placement count:5 2024-03-21T12:46:38.438 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:38 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' 2024-03-21T12:46:39.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-21T12:46:39.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.805+0000 7ff6898da700 1 Processor -- start 2024-03-21T12:46:39.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.805+0000 7ff6898da700 1 -- start start 2024-03-21T12:46:39.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.805+0000 7ff6898da700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff684109320 con 0x7ff684104e70 2024-03-21T12:46:39.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.805+0000 7ff682ffd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff684104e70 0x7ff684105290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52748/0 (socket says 172.21.15.43:52748) 2024-03-21T12:46:39.753 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.805+0000 7ff682ffd700 1 -- 172.21.15.43:0/2686445402 learned_addr learned my addr 172.21.15.43:0/2686445402 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.806+0000 7ff681ffb700 1 -- 172.21.15.43:0/2686445402 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935374508 0 0) 0x7ff684109320 con 0x7ff684104e70 2024-03-21T12:46:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.806+0000 7ff681ffb700 1 -- 172.21.15.43:0/2686445402 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff66c0035b0 con 0x7ff684104e70 2024-03-21T12:46:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.806+0000 7ff681ffb700 1 -- 172.21.15.43:0/2686445402 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 632147833 0 0) 0x7ff66c0035b0 con 0x7ff684104e70 2024-03-21T12:46:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.806+0000 7ff681ffb700 1 -- 172.21.15.43:0/2686445402 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68410a500 con 0x7ff684104e70 2024-03-21T12:46:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.806+0000 7ff681ffb700 1 -- 172.21.15.43:0/2686445402 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7ff674001580 con 0x7ff684104e70 2024-03-21T12:46:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.806+0000 7ff681ffb700 1 -- 172.21.15.43:0/2686445402 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7ff674001d20 con 0x7ff684104e70 2024-03-21T12:46:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.806+0000 7ff681ffb700 1 -- 172.21.15.43:0/2686445402 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7ff674005960 con 0x7ff684104e70 2024-03-21T12:46:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.806+0000 7ff6898da700 1 -- 172.21.15.43:0/2686445402 >> v1:172.21.15.43:6789/0 conn(0x7ff684104e70 legacy=0x7ff684105290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.807+0000 7ff6898da700 1 -- 172.21.15.43:0/2686445402 shutdown_connections 2024-03-21T12:46:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.807+0000 7ff6898da700 1 -- 172.21.15.43:0/2686445402 >> 172.21.15.43:0/2686445402 conn(0x7ff684100480 msgr2=0x7ff6841028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.807+0000 7ff6898da700 1 -- 172.21.15.43:0/2686445402 shutdown_connections 2024-03-21T12:46:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.807+0000 7ff6898da700 1 -- 172.21.15.43:0/2686445402 wait complete. 2024-03-21T12:46:39.754 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.807+0000 7ff6898da700 1 Processor -- start 2024-03-21T12:46:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.807+0000 7ff6898da700 1 -- start start 2024-03-21T12:46:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.807+0000 7ff6898da700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff68419faa0 con 0x7ff684104e70 2024-03-21T12:46:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.808+0000 7ff682ffd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff684104e70 0x7ff6841022f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52750/0 (socket says 172.21.15.43:52750) 2024-03-21T12:46:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.808+0000 7ff682ffd700 1 -- 172.21.15.43:0/2204101879 learned_addr learned my addr 172.21.15.43:0/2204101879 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.808+0000 7ff6888d8700 1 -- 172.21.15.43:0/2204101879 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1184514208 0 0) 0x7ff68419faa0 con 0x7ff684104e70 2024-03-21T12:46:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.808+0000 7ff6888d8700 1 -- 172.21.15.43:0/2204101879 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff66c0055f0 con 0x7ff684104e70 2024-03-21T12:46:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.808+0000 7ff6888d8700 1 -- 172.21.15.43:0/2204101879 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2314892455 0 0) 0x7ff66c0055f0 con 0x7ff684104e70 2024-03-21T12:46:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.808+0000 7ff6888d8700 1 -- 172.21.15.43:0/2204101879 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff68419faa0 con 0x7ff684104e70 2024-03-21T12:46:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.808+0000 7ff6888d8700 1 -- 172.21.15.43:0/2204101879 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7ff674001080 con 0x7ff684104e70 2024-03-21T12:46:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.808+0000 7ff6888d8700 1 -- 172.21.15.43:0/2204101879 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3889047966 0 0) 0x7ff68419faa0 con 0x7ff684104e70 2024-03-21T12:46:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.808+0000 7ff6888d8700 1 -- 172.21.15.43:0/2204101879 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6841a0c80 con 0x7ff684104e70 2024-03-21T12:46:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.808+0000 7ff6898da700 1 -- 172.21.15.43:0/2204101879 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68419fcd0 con 0x7ff684104e70 2024-03-21T12:46:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.808+0000 7ff6898da700 1 -- 172.21.15.43:0/2204101879 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6841a0210 con 0x7ff684104e70 2024-03-21T12:46:39.755 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.808+0000 7ff6888d8700 1 -- 172.21.15.43:0/2204101879 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1037+0+0 (unknown 3161219909 0 0) 0x7ff674001cc0 con 0x7ff684104e70 2024-03-21T12:46:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.808+0000 7ff6888d8700 1 -- 172.21.15.43:0/2204101879 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7ff674006010 con 0x7ff684104e70 2024-03-21T12:46:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.809+0000 7ff6888d8700 1 -- 172.21.15.43:0/2204101879 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 8) v1 ==== 46259+0+0 (unknown 94073229 0 0) 0x7ff674011670 con 0x7ff684104e70 2024-03-21T12:46:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.809+0000 7ff6898da700 1 -- 172.21.15.43:0/2204101879 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff670005150 con 0x7ff684104e70 2024-03-21T12:46:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.809+0000 7ff6888d8700 1 -- 172.21.15.43:0/2204101879 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 73304150 0 0) 0x7ff674047980 con 0x7ff684104e70 2024-03-21T12:46:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:38.814+0000 7ff6888d8700 1 -- 172.21.15.43:0/2204101879 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7ff674017580 con 0x7ff684104e70 2024-03-21T12:46:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:39.037+0000 7ff6898da700 1 -- 172.21.15.43:0/2204101879 --> v1:172.21.15.43:6800/1388905519 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7ff670000bc0 con 0x7ff66c03a6b0 2024-03-21T12:46:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:39.042+0000 7ff6888d8700 1 -- 172.21.15.43:0/2204101879 <== mgr.14120 v1:172.21.15.43:6800/1388905519 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 325935098) 0x7ff670000bc0 con 0x7ff66c03a6b0 2024-03-21T12:46:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:39.044+0000 7ff6898da700 1 -- 172.21.15.43:0/2204101879 >> v1:172.21.15.43:6800/1388905519 conn(0x7ff66c03a6b0 legacy=0x7ff66c03cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:39.044+0000 7ff6898da700 1 -- 172.21.15.43:0/2204101879 >> v1:172.21.15.43:6789/0 conn(0x7ff684104e70 legacy=0x7ff6841022f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:39.044+0000 7ff6898da700 1 -- 172.21.15.43:0/2204101879 shutdown_connections 2024-03-21T12:46:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:39.044+0000 7ff6898da700 1 -- 172.21.15.43:0/2204101879 >> 172.21.15.43:0/2204101879 conn(0x7ff684100480 msgr2=0x7ff684108ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:39.044+0000 7ff6898da700 1 -- 172.21.15.43:0/2204101879 shutdown_connections 2024-03-21T12:46:39.756 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:39.044+0000 7ff6898da700 1 -- 172.21.15.43:0/2204101879 wait complete. 2024-03-21T12:46:40.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:40 smithi043 ceph-mon[30062]: from='client.14144 v1:172.21.15.43:0/2204101879' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:46:40.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:40 smithi043 ceph-mon[30062]: Saving service mgr spec with placement count:2 2024-03-21T12:46:40.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:40 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' 2024-03-21T12:46:40.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:40 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' 2024-03-21T12:46:40.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:40 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' 2024-03-21T12:46:41.370 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.250+0000 7fe33e10a700 1 Processor -- start 2024-03-21T12:46:41.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.250+0000 7fe33e10a700 1 -- start start 2024-03-21T12:46:41.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.250+0000 7fe33e10a700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe3380796f0 con 0x7fe338106a90 2024-03-21T12:46:41.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.250+0000 7fe3377fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe338106a90 0x7fe338108e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52760/0 (socket says 172.21.15.43:52760) 2024-03-21T12:46:41.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.250+0000 7fe3377fe700 1 -- 172.21.15.43:0/3686236435 learned_addr learned my addr 172.21.15.43:0/3686236435 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:41.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.250+0000 7fe3367fc700 1 -- 172.21.15.43:0/3686236435 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1759448990 0 0) 0x7fe3380796f0 con 0x7fe338106a90 2024-03-21T12:46:41.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.251+0000 7fe3367fc700 1 -- 172.21.15.43:0/3686236435 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3200035b0 con 0x7fe338106a90 2024-03-21T12:46:41.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.251+0000 7fe3367fc700 1 -- 172.21.15.43:0/3686236435 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3605645673 0 0) 0x7fe3200035b0 con 0x7fe338106a90 2024-03-21T12:46:41.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.251+0000 7fe3367fc700 1 -- 172.21.15.43:0/3686236435 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe33810b5b0 con 0x7fe338106a90 2024-03-21T12:46:41.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.251+0000 7fe3367fc700 1 -- 172.21.15.43:0/3686236435 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fe328001580 con 0x7fe338106a90 2024-03-21T12:46:41.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.251+0000 7fe3367fc700 1 -- 172.21.15.43:0/3686236435 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7fe328004930 con 0x7fe338106a90 2024-03-21T12:46:41.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.251+0000 7fe3367fc700 1 -- 172.21.15.43:0/3686236435 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fe328005a70 con 0x7fe338106a90 2024-03-21T12:46:41.371 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.251+0000 7fe33e10a700 1 -- 172.21.15.43:0/3686236435 >> v1:172.21.15.43:6789/0 conn(0x7fe338106a90 legacy=0x7fe338108e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:41.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.252+0000 7fe33e10a700 1 -- 172.21.15.43:0/3686236435 shutdown_connections 2024-03-21T12:46:41.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.252+0000 7fe33e10a700 1 -- 172.21.15.43:0/3686236435 >> 172.21.15.43:0/3686236435 conn(0x7fe3381004c0 msgr2=0x7fe3381028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:41.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.252+0000 7fe33e10a700 1 -- 172.21.15.43:0/3686236435 shutdown_connections 2024-03-21T12:46:41.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.252+0000 7fe33e10a700 1 -- 172.21.15.43:0/3686236435 wait complete. 2024-03-21T12:46:41.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.252+0000 7fe33e10a700 1 Processor -- start 2024-03-21T12:46:41.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.252+0000 7fe33e10a700 1 -- start start 2024-03-21T12:46:41.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.253+0000 7fe33e10a700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe33819fab0 con 0x7fe338106a90 2024-03-21T12:46:41.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.253+0000 7fe3377fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe338106a90 0x7fe338106310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52764/0 (socket says 172.21.15.43:52764) 2024-03-21T12:46:41.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.253+0000 7fe3377fe700 1 -- 172.21.15.43:0/1907888235 learned_addr learned my addr 172.21.15.43:0/1907888235 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:41.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.253+0000 7fe334ff9700 1 -- 172.21.15.43:0/1907888235 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2245851418 0 0) 0x7fe33819fab0 con 0x7fe338106a90 2024-03-21T12:46:41.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.253+0000 7fe334ff9700 1 -- 172.21.15.43:0/1907888235 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3200055f0 con 0x7fe338106a90 2024-03-21T12:46:41.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.253+0000 7fe334ff9700 1 -- 172.21.15.43:0/1907888235 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2079856576 0 0) 0x7fe3200055f0 con 0x7fe338106a90 2024-03-21T12:46:41.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.253+0000 7fe334ff9700 1 -- 172.21.15.43:0/1907888235 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe33819fab0 con 0x7fe338106a90 2024-03-21T12:46:41.372 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.253+0000 7fe334ff9700 1 -- 172.21.15.43:0/1907888235 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fe328001080 con 0x7fe338106a90 2024-03-21T12:46:41.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.253+0000 7fe334ff9700 1 -- 172.21.15.43:0/1907888235 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3862644723 0 0) 0x7fe33819fab0 con 0x7fe338106a90 2024-03-21T12:46:41.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.253+0000 7fe334ff9700 1 -- 172.21.15.43:0/1907888235 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3381a0c90 con 0x7fe338106a90 2024-03-21T12:46:41.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.253+0000 7fe33e10a700 1 -- 172.21.15.43:0/1907888235 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe33819fce0 con 0x7fe338106a90 2024-03-21T12:46:41.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.253+0000 7fe33e10a700 1 -- 172.21.15.43:0/1907888235 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3381a0220 con 0x7fe338106a90 2024-03-21T12:46:41.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.254+0000 7fe334ff9700 1 -- 172.21.15.43:0/1907888235 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7fe3280059e0 con 0x7fe338106a90 2024-03-21T12:46:41.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.254+0000 7fe334ff9700 1 -- 172.21.15.43:0/1907888235 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fe328006160 con 0x7fe338106a90 2024-03-21T12:46:41.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.254+0000 7fe334ff9700 1 -- 172.21.15.43:0/1907888235 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 8) v1 ==== 46259+0+0 (unknown 94073229 0 0) 0x7fe3280117c0 con 0x7fe338106a90 2024-03-21T12:46:41.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.255+0000 7fe334ff9700 1 -- 172.21.15.43:0/1907888235 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 73304150 0 0) 0x7fe3280488e0 con 0x7fe338106a90 2024-03-21T12:46:41.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.255+0000 7fe33e10a700 1 -- 172.21.15.43:0/1907888235 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe324005150 con 0x7fe338106a90 2024-03-21T12:46:41.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.259+0000 7fe334ff9700 1 -- 172.21.15.43:0/1907888235 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7fe328017600 con 0x7fe338106a90 2024-03-21T12:46:41.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.490+0000 7fe33e10a700 1 -- 172.21.15.43:0/1907888235 --> v1:172.21.15.43:6789/0 -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fe324005440 con 0x7fe338106a90 2024-03-21T12:46:41.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.492+0000 7fe334ff9700 1 -- 172.21.15.43:0/1907888235 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (unknown 2684808024 0 0) 0x7fe328016f00 con 0x7fe338106a90 2024-03-21T12:46:41.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.494+0000 7fe33e10a700 1 -- 172.21.15.43:0/1907888235 >> v1:172.21.15.43:6800/1388905519 conn(0x7fe32003a6c0 legacy=0x7fe32003cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:41.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.494+0000 7fe33e10a700 1 -- 172.21.15.43:0/1907888235 >> v1:172.21.15.43:6789/0 conn(0x7fe338106a90 legacy=0x7fe338106310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:41.373 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.494+0000 7fe33e10a700 1 -- 172.21.15.43:0/1907888235 shutdown_connections 2024-03-21T12:46:41.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.494+0000 7fe33e10a700 1 -- 172.21.15.43:0/1907888235 >> 172.21.15.43:0/1907888235 conn(0x7fe3381004c0 msgr2=0x7fe338190840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:41.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.495+0000 7fe33e10a700 1 -- 172.21.15.43:0/1907888235 shutdown_connections 2024-03-21T12:46:41.374 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:40.495+0000 7fe33e10a700 1 -- 172.21.15.43:0/1907888235 wait complete. 2024-03-21T12:46:41.617 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1907888235' entity='client.admin' 2024-03-21T12:46:43.086 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.294+0000 7f7de65f3700 1 Processor -- start 2024-03-21T12:46:43.086 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.294+0000 7f7de65f3700 1 -- start start 2024-03-21T12:46:43.086 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.295+0000 7f7de65f3700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7de0079fa0 con 0x7f7de0106ab0 2024-03-21T12:46:43.086 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.295+0000 7f7ddffff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7de0106ab0 0x7f7de0108ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38218/0 (socket says 172.21.15.43:38218) 2024-03-21T12:46:43.087 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.295+0000 7f7ddffff700 1 -- 172.21.15.43:0/3960939081 learned_addr learned my addr 172.21.15.43:0/3960939081 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:43.087 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.295+0000 7f7ddeffd700 1 -- 172.21.15.43:0/3960939081 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3686003586 0 0) 0x7f7de0079fa0 con 0x7f7de0106ab0 2024-03-21T12:46:43.087 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.295+0000 7f7ddeffd700 1 -- 172.21.15.43:0/3960939081 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dcc0035b0 con 0x7f7de0106ab0 2024-03-21T12:46:43.087 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.295+0000 7f7ddeffd700 1 -- 172.21.15.43:0/3960939081 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3814369722 0 0) 0x7f7dcc0035b0 con 0x7f7de0106ab0 2024-03-21T12:46:43.087 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.295+0000 7f7ddeffd700 1 -- 172.21.15.43:0/3960939081 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de010b5d0 con 0x7f7de0106ab0 2024-03-21T12:46:43.087 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.295+0000 7f7ddeffd700 1 -- 172.21.15.43:0/3960939081 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f7dd0001580 con 0x7f7de0106ab0 2024-03-21T12:46:43.087 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.295+0000 7f7ddeffd700 1 -- 172.21.15.43:0/3960939081 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f7dd0004930 con 0x7f7de0106ab0 2024-03-21T12:46:43.087 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.295+0000 7f7ddeffd700 1 -- 172.21.15.43:0/3960939081 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f7dd0005a70 con 0x7f7de0106ab0 2024-03-21T12:46:43.087 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.296+0000 7f7de65f3700 1 -- 172.21.15.43:0/3960939081 >> v1:172.21.15.43:6789/0 conn(0x7f7de0106ab0 legacy=0x7f7de0108ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:43.087 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.296+0000 7f7de65f3700 1 -- 172.21.15.43:0/3960939081 shutdown_connections 2024-03-21T12:46:43.087 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.296+0000 7f7de65f3700 1 -- 172.21.15.43:0/3960939081 >> 172.21.15.43:0/3960939081 conn(0x7f7de01004c0 msgr2=0x7f7de01028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:43.087 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.296+0000 7f7de65f3700 1 -- 172.21.15.43:0/3960939081 shutdown_connections 2024-03-21T12:46:43.087 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.296+0000 7f7de65f3700 1 -- 172.21.15.43:0/3960939081 wait complete. 2024-03-21T12:46:43.087 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.297+0000 7f7de65f3700 1 Processor -- start 2024-03-21T12:46:43.087 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.297+0000 7f7de65f3700 1 -- start start 2024-03-21T12:46:43.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.297+0000 7f7de65f3700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7de019fb30 con 0x7f7de0106ab0 2024-03-21T12:46:43.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.297+0000 7f7ddffff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7de0106ab0 0x7f7de0106330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38232/0 (socket says 172.21.15.43:38232) 2024-03-21T12:46:43.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.297+0000 7f7ddffff700 1 -- 172.21.15.43:0/1965869472 learned_addr learned my addr 172.21.15.43:0/1965869472 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:43.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.297+0000 7f7ddd7fa700 1 -- 172.21.15.43:0/1965869472 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3750550024 0 0) 0x7f7de019fb30 con 0x7f7de0106ab0 2024-03-21T12:46:43.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.297+0000 7f7ddd7fa700 1 -- 172.21.15.43:0/1965869472 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7dcc0055f0 con 0x7f7de0106ab0 2024-03-21T12:46:43.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.298+0000 7f7ddd7fa700 1 -- 172.21.15.43:0/1965869472 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2995106966 0 0) 0x7f7dcc0055f0 con 0x7f7de0106ab0 2024-03-21T12:46:43.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.298+0000 7f7ddd7fa700 1 -- 172.21.15.43:0/1965869472 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7de019fb30 con 0x7f7de0106ab0 2024-03-21T12:46:43.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.298+0000 7f7ddd7fa700 1 -- 172.21.15.43:0/1965869472 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f7dd0001080 con 0x7f7de0106ab0 2024-03-21T12:46:43.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.298+0000 7f7ddd7fa700 1 -- 172.21.15.43:0/1965869472 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4279775581 0 0) 0x7f7de019fb30 con 0x7f7de0106ab0 2024-03-21T12:46:43.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.298+0000 7f7ddd7fa700 1 -- 172.21.15.43:0/1965869472 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7de01a0d10 con 0x7f7de0106ab0 2024-03-21T12:46:43.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.298+0000 7f7de65f3700 1 -- 172.21.15.43:0/1965869472 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de019fd60 con 0x7f7de0106ab0 2024-03-21T12:46:43.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.298+0000 7f7de65f3700 1 -- 172.21.15.43:0/1965869472 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de01a02a0 con 0x7f7de0106ab0 2024-03-21T12:46:43.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.298+0000 7f7ddd7fa700 1 -- 172.21.15.43:0/1965869472 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f7dd0004930 con 0x7f7de0106ab0 2024-03-21T12:46:43.088 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.298+0000 7f7ddd7fa700 1 -- 172.21.15.43:0/1965869472 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f7dd0006050 con 0x7f7de0106ab0 2024-03-21T12:46:43.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.299+0000 7f7ddd7fa700 1 -- 172.21.15.43:0/1965869472 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 8) v1 ==== 46259+0+0 (unknown 94073229 0 0) 0x7f7dd0011730 con 0x7f7de0106ab0 2024-03-21T12:46:43.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.299+0000 7f7ddd7fa700 1 -- 172.21.15.43:0/1965869472 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 73304150 0 0) 0x7f7dd0048900 con 0x7f7de0106ab0 2024-03-21T12:46:43.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.299+0000 7f7de65f3700 1 -- 172.21.15.43:0/1965869472 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc0005150 con 0x7f7de0106ab0 2024-03-21T12:46:43.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.304+0000 7f7ddd7fa700 1 -- 172.21.15.43:0/1965869472 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f7dd0017620 con 0x7f7de0106ab0 2024-03-21T12:46:43.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.510+0000 7f7de65f3700 1 -- 172.21.15.43:0/1965869472 --> v1:172.21.15.43:6789/0 -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f7dc0005440 con 0x7f7de0106ab0 2024-03-21T12:46:43.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.512+0000 7f7ddd7fa700 1 -- 172.21.15.43:0/1965869472 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (unknown 1496729260 0 0) 0x7f7dd0016f20 con 0x7f7de0106ab0 2024-03-21T12:46:43.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.514+0000 7f7de65f3700 1 -- 172.21.15.43:0/1965869472 >> v1:172.21.15.43:6800/1388905519 conn(0x7f7dcc03a670 legacy=0x7f7dcc03cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:43.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.514+0000 7f7de65f3700 1 -- 172.21.15.43:0/1965869472 >> v1:172.21.15.43:6789/0 conn(0x7f7de0106ab0 legacy=0x7f7de0106330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:43.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.515+0000 7f7de65f3700 1 -- 172.21.15.43:0/1965869472 shutdown_connections 2024-03-21T12:46:43.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.515+0000 7f7de65f3700 1 -- 172.21.15.43:0/1965869472 >> 172.21.15.43:0/1965869472 conn(0x7f7de01004c0 msgr2=0x7f7de0190860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:43.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.515+0000 7f7de65f3700 1 -- 172.21.15.43:0/1965869472 shutdown_connections 2024-03-21T12:46:43.089 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:42.515+0000 7f7de65f3700 1 -- 172.21.15.43:0/1965869472 wait complete. 2024-03-21T12:46:43.089 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling the dashboard module... 2024-03-21T12:46:43.349 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:43 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' 2024-03-21T12:46:43.349 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1965869472' entity='client.admin' 2024-03-21T12:46:44.983 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:44 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ignoring --setuser ceph since I am not root 2024-03-21T12:46:44.984 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:44 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ignoring --setgroup ceph since I am not root 2024-03-21T12:46:44.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:44 smithi043 ceph-mon[30062]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:46:44.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:44 smithi043 ceph-mon[30062]: from='mgr.14120 v1:172.21.15.43:0/1923146123' entity='mgr.y' 2024-03-21T12:46:44.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:44 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/793361618' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-21T12:46:45.575 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:45 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:45.394+0000 7efd4d04e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T12:46:45.575 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:45 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:45.489+0000 7efd4d04e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T12:46:45.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.597+0000 7ffb0dbe8700 1 Processor -- start 2024-03-21T12:46:45.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.598+0000 7ffb0dbe8700 1 -- start start 2024-03-21T12:46:45.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.598+0000 7ffb0dbe8700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ffb0810a430 con 0x7ffb08106000 2024-03-21T12:46:45.580 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.598+0000 7ffb077fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ffb08106000 0x7ffb08106420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38242/0 (socket says 172.21.15.43:38242) 2024-03-21T12:46:45.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.598+0000 7ffb077fe700 1 -- 172.21.15.43:0/3379725180 learned_addr learned my addr 172.21.15.43:0/3379725180 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:45.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.598+0000 7ffb067fc700 1 -- 172.21.15.43:0/3379725180 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 188669203 0 0) 0x7ffb0810a430 con 0x7ffb08106000 2024-03-21T12:46:45.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.598+0000 7ffb067fc700 1 -- 172.21.15.43:0/3379725180 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaf40035b0 con 0x7ffb08106000 2024-03-21T12:46:45.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.599+0000 7ffb067fc700 1 -- 172.21.15.43:0/3379725180 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3825080624 0 0) 0x7ffaf40035b0 con 0x7ffb08106000 2024-03-21T12:46:45.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.599+0000 7ffb067fc700 1 -- 172.21.15.43:0/3379725180 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0810b610 con 0x7ffb08106000 2024-03-21T12:46:45.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.599+0000 7ffb067fc700 1 -- 172.21.15.43:0/3379725180 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7ffaf0001580 con 0x7ffb08106000 2024-03-21T12:46:45.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.599+0000 7ffb067fc700 1 -- 172.21.15.43:0/3379725180 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7ffaf0004930 con 0x7ffb08106000 2024-03-21T12:46:45.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.599+0000 7ffb067fc700 1 -- 172.21.15.43:0/3379725180 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7ffaf0005960 con 0x7ffb08106000 2024-03-21T12:46:45.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.599+0000 7ffb0dbe8700 1 -- 172.21.15.43:0/3379725180 >> v1:172.21.15.43:6789/0 conn(0x7ffb08106000 legacy=0x7ffb08106420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:45.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.599+0000 7ffb0dbe8700 1 -- 172.21.15.43:0/3379725180 shutdown_connections 2024-03-21T12:46:45.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.600+0000 7ffb0dbe8700 1 -- 172.21.15.43:0/3379725180 >> 172.21.15.43:0/3379725180 conn(0x7ffb081015e0 msgr2=0x7ffb08103a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:45.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.600+0000 7ffb0dbe8700 1 -- 172.21.15.43:0/3379725180 shutdown_connections 2024-03-21T12:46:45.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.600+0000 7ffb0dbe8700 1 -- 172.21.15.43:0/3379725180 wait complete. 2024-03-21T12:46:45.581 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.600+0000 7ffb0dbe8700 1 Processor -- start 2024-03-21T12:46:45.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.600+0000 7ffb0dbe8700 1 -- start start 2024-03-21T12:46:45.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.600+0000 7ffb0dbe8700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ffb0819fae0 con 0x7ffb08106000 2024-03-21T12:46:45.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.601+0000 7ffb077fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ffb08106000 0x7ffb0807bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38258/0 (socket says 172.21.15.43:38258) 2024-03-21T12:46:45.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.601+0000 7ffb077fe700 1 -- 172.21.15.43:0/793361618 learned_addr learned my addr 172.21.15.43:0/793361618 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:45.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.601+0000 7ffb04ff9700 1 -- 172.21.15.43:0/793361618 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1370663943 0 0) 0x7ffb0819fae0 con 0x7ffb08106000 2024-03-21T12:46:45.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.601+0000 7ffb04ff9700 1 -- 172.21.15.43:0/793361618 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaf40055f0 con 0x7ffb08106000 2024-03-21T12:46:45.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.601+0000 7ffb04ff9700 1 -- 172.21.15.43:0/793361618 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1980829092 0 0) 0x7ffaf40055f0 con 0x7ffb08106000 2024-03-21T12:46:45.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.601+0000 7ffb04ff9700 1 -- 172.21.15.43:0/793361618 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffb0819fae0 con 0x7ffb08106000 2024-03-21T12:46:45.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.601+0000 7ffb04ff9700 1 -- 172.21.15.43:0/793361618 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7ffaf0001080 con 0x7ffb08106000 2024-03-21T12:46:45.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.601+0000 7ffb04ff9700 1 -- 172.21.15.43:0/793361618 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1090557833 0 0) 0x7ffb0819fae0 con 0x7ffb08106000 2024-03-21T12:46:45.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.601+0000 7ffb04ff9700 1 -- 172.21.15.43:0/793361618 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb081a0cc0 con 0x7ffb08106000 2024-03-21T12:46:45.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.601+0000 7ffb0dbe8700 1 -- 172.21.15.43:0/793361618 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0819fd10 con 0x7ffb08106000 2024-03-21T12:46:45.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.601+0000 7ffb0dbe8700 1 -- 172.21.15.43:0/793361618 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb081a0250 con 0x7ffb08106000 2024-03-21T12:46:45.582 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.601+0000 7ffb04ff9700 1 -- 172.21.15.43:0/793361618 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7ffaf0001540 con 0x7ffb08106000 2024-03-21T12:46:45.583 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.601+0000 7ffb04ff9700 1 -- 172.21.15.43:0/793361618 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7ffaf0006320 con 0x7ffb08106000 2024-03-21T12:46:45.583 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.602+0000 7ffb04ff9700 1 -- 172.21.15.43:0/793361618 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 8) v1 ==== 46259+0+0 (unknown 94073229 0 0) 0x7ffaf0011980 con 0x7ffb08106000 2024-03-21T12:46:45.583 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.602+0000 7ffb0dbe8700 1 -- 172.21.15.43:0/793361618 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae8005150 con 0x7ffb08106000 2024-03-21T12:46:45.583 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.602+0000 7ffb04ff9700 1 -- 172.21.15.43:0/793361618 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 73304150 0 0) 0x7ffaf00479e0 con 0x7ffb08106000 2024-03-21T12:46:45.583 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.607+0000 7ffb04ff9700 1 -- 172.21.15.43:0/793361618 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7ffaf00175b0 con 0x7ffb08106000 2024-03-21T12:46:45.583 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:43.875+0000 7ffb0dbe8700 1 -- 172.21.15.43:0/793361618 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7ffae8005440 con 0x7ffb08106000 2024-03-21T12:46:45.583 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:44.811+0000 7ffb04ff9700 1 -- 172.21.15.43:0/793361618 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (unknown 1498667528 0 0) 0x7ffaf0016eb0 con 0x7ffb08106000 2024-03-21T12:46:45.583 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:44.812+0000 7ffb04ff9700 1 -- 172.21.15.43:0/793361618 <== mon.0 v1:172.21.15.43:6789/0 11 ==== mgrmap(e 9) v1 ==== 46272+0+0 (unknown 1750918330 0 0) 0x7ffaf0046420 con 0x7ffb08106000 2024-03-21T12:46:45.583 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:44.814+0000 7ffb0dbe8700 1 -- 172.21.15.43:0/793361618 >> v1:172.21.15.43:6800/1388905519 conn(0x7ffaf403a700 legacy=0x7ffaf403cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:45.583 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:44.814+0000 7ffb0dbe8700 1 -- 172.21.15.43:0/793361618 >> v1:172.21.15.43:6789/0 conn(0x7ffb08106000 legacy=0x7ffb0807bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:45.583 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:44.814+0000 7ffb0dbe8700 1 -- 172.21.15.43:0/793361618 shutdown_connections 2024-03-21T12:46:45.583 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:44.814+0000 7ffb0dbe8700 1 -- 172.21.15.43:0/793361618 >> 172.21.15.43:0/793361618 conn(0x7ffb081015e0 msgr2=0x7ffb081039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:45.583 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:44.814+0000 7ffb0dbe8700 1 -- 172.21.15.43:0/793361618 shutdown_connections 2024-03-21T12:46:45.583 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:44.814+0000 7ffb0dbe8700 1 -- 172.21.15.43:0/793361618 wait complete. 2024-03-21T12:46:45.919 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/793361618' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-21T12:46:45.920 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:45 smithi043 ceph-mon[30062]: mgrmap e9: y(active, since 21s) 2024-03-21T12:46:46.219 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:46 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:46.167+0000 7efd4d04e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T12:46:46.482 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:46 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:46.263+0000 7efd4d04e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T12:46:46.482 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:46 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:46.407+0000 7efd4d04e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T12:46:46.733 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:46 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:46.508+0000 7efd4d04e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T12:46:46.733 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:46 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:46.595+0000 7efd4d04e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T12:46:47.070 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3014835107' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-21T12:46:47.070 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:46 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:46.867+0000 7efd4d04e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T12:46:47.070 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:46 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:46.980+0000 7efd4d04e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T12:46:47.070 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:47 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:47.068+0000 7efd4d04e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T12:46:47.340 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:47 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:47.167+0000 7efd4d04e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T12:46:47.352 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-03-21T12:46:47.352 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-21T12:46:47.352 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-21T12:46:47.352 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-03-21T12:46:47.352 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-21T12:46:47.352 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-03-21T12:46:47.353 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.481+0000 7f0c9447c700 1 Processor -- start 2024-03-21T12:46:47.353 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.481+0000 7f0c9447c700 1 -- start start 2024-03-21T12:46:47.353 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.481+0000 7f0c9447c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0c8c0796d0 con 0x7f0c8c1048a0 2024-03-21T12:46:47.353 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.481+0000 7f0c92218700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0c8c1048a0 0x7f0c8c104cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38294/0 (socket says 172.21.15.43:38294) 2024-03-21T12:46:47.353 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.481+0000 7f0c92218700 1 -- 172.21.15.43:0/2240372856 learned_addr learned my addr 172.21.15.43:0/2240372856 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:47.354 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.481+0000 7f0c91216700 1 -- 172.21.15.43:0/2240372856 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3892138868 0 0) 0x7f0c8c0796d0 con 0x7f0c8c1048a0 2024-03-21T12:46:47.354 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.481+0000 7f0c91216700 1 -- 172.21.15.43:0/2240372856 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c780035b0 con 0x7f0c8c1048a0 2024-03-21T12:46:47.354 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.482+0000 7f0c91216700 1 -- 172.21.15.43:0/2240372856 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2485666291 0 0) 0x7f0c780035b0 con 0x7f0c8c1048a0 2024-03-21T12:46:47.354 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.482+0000 7f0c91216700 1 -- 172.21.15.43:0/2240372856 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8c1063e0 con 0x7f0c8c1048a0 2024-03-21T12:46:47.354 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.482+0000 7f0c91216700 1 -- 172.21.15.43:0/2240372856 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f0c88001580 con 0x7f0c8c1048a0 2024-03-21T12:46:47.354 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.482+0000 7f0c91216700 1 -- 172.21.15.43:0/2240372856 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f0c88004930 con 0x7f0c8c1048a0 2024-03-21T12:46:47.354 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.482+0000 7f0c91216700 1 -- 172.21.15.43:0/2240372856 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f0c88005a70 con 0x7f0c8c1048a0 2024-03-21T12:46:47.355 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.482+0000 7f0c9447c700 1 -- 172.21.15.43:0/2240372856 >> v1:172.21.15.43:6789/0 conn(0x7f0c8c1048a0 legacy=0x7f0c8c104cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:47.355 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.483+0000 7f0c9447c700 1 -- 172.21.15.43:0/2240372856 shutdown_connections 2024-03-21T12:46:47.355 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.483+0000 7f0c9447c700 1 -- 172.21.15.43:0/2240372856 >> 172.21.15.43:0/2240372856 conn(0x7f0c8c1004c0 msgr2=0x7f0c8c1028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:47.355 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.483+0000 7f0c9447c700 1 -- 172.21.15.43:0/2240372856 shutdown_connections 2024-03-21T12:46:47.355 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.483+0000 7f0c9447c700 1 -- 172.21.15.43:0/2240372856 wait complete. 2024-03-21T12:46:47.355 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.483+0000 7f0c9447c700 1 Processor -- start 2024-03-21T12:46:47.355 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.483+0000 7f0c9447c700 1 -- start start 2024-03-21T12:46:47.356 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.483+0000 7f0c9447c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0c8c19c3b0 con 0x7f0c8c1048a0 2024-03-21T12:46:47.356 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.484+0000 7f0c92218700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0c8c1048a0 0x7f0c8c19bca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38310/0 (socket says 172.21.15.43:38310) 2024-03-21T12:46:47.356 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.484+0000 7f0c92218700 1 -- 172.21.15.43:0/3014835107 learned_addr learned my addr 172.21.15.43:0/3014835107 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:47.356 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.484+0000 7f0c7f7fe700 1 -- 172.21.15.43:0/3014835107 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2577801016 0 0) 0x7f0c8c19c3b0 con 0x7f0c8c1048a0 2024-03-21T12:46:47.356 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.484+0000 7f0c7f7fe700 1 -- 172.21.15.43:0/3014835107 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c780055f0 con 0x7f0c8c1048a0 2024-03-21T12:46:47.356 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.484+0000 7f0c7f7fe700 1 -- 172.21.15.43:0/3014835107 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2679774675 0 0) 0x7f0c780055f0 con 0x7f0c8c1048a0 2024-03-21T12:46:47.357 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.484+0000 7f0c7f7fe700 1 -- 172.21.15.43:0/3014835107 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0c8c19c3b0 con 0x7f0c8c1048a0 2024-03-21T12:46:47.357 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.484+0000 7f0c7f7fe700 1 -- 172.21.15.43:0/3014835107 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f0c88001080 con 0x7f0c8c1048a0 2024-03-21T12:46:47.357 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.484+0000 7f0c7f7fe700 1 -- 172.21.15.43:0/3014835107 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 161930429 0 0) 0x7f0c8c19c3b0 con 0x7f0c8c1048a0 2024-03-21T12:46:47.357 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.484+0000 7f0c7f7fe700 1 -- 172.21.15.43:0/3014835107 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8c19c580 con 0x7f0c8c1048a0 2024-03-21T12:46:47.357 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.484+0000 7f0c9447c700 1 -- 172.21.15.43:0/3014835107 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8c19c870 con 0x7f0c8c1048a0 2024-03-21T12:46:47.357 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.484+0000 7f0c9447c700 1 -- 172.21.15.43:0/3014835107 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c8c1a03c0 con 0x7f0c8c1048a0 2024-03-21T12:46:47.358 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.484+0000 7f0c7f7fe700 1 -- 172.21.15.43:0/3014835107 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f0c88004930 con 0x7f0c8c1048a0 2024-03-21T12:46:47.358 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.484+0000 7f0c7f7fe700 1 -- 172.21.15.43:0/3014835107 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f0c88006020 con 0x7f0c8c1048a0 2024-03-21T12:46:47.358 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.485+0000 7f0c7f7fe700 1 -- 172.21.15.43:0/3014835107 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 9) v1 ==== 46272+0+0 (unknown 1750918330 0 0) 0x7f0c88011690 con 0x7f0c8c1048a0 2024-03-21T12:46:47.358 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.485+0000 7f0c9447c700 1 -- 172.21.15.43:0/3014835107 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c70005120 con 0x7f0c8c1048a0 2024-03-21T12:46:47.358 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.485+0000 7f0c7f7fe700 1 -- 172.21.15.43:0/3014835107 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 73304150 0 0) 0x7f0c88047a70 con 0x7f0c8c1048a0 2024-03-21T12:46:47.359 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.485+0000 7f0c91a17700 1 -- 172.21.15.43:0/3014835107 >> v1:172.21.15.43:6800/1388905519 conn(0x7f0c7803a750 legacy=0x7f0c7803cc10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.43:6800/1388905519 2024-03-21T12:46:47.359 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.490+0000 7f0c7f7fe700 1 -- 172.21.15.43:0/3014835107 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f0c880175f0 con 0x7f0c8c1048a0 2024-03-21T12:46:47.359 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.686+0000 7f0c91a17700 1 -- 172.21.15.43:0/3014835107 >> v1:172.21.15.43:6800/1388905519 conn(0x7f0c7803a750 legacy=0x7f0c7803cc10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.43:6800/1388905519 2024-03-21T12:46:47.359 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.754+0000 7f0c9447c700 1 -- 172.21.15.43:0/3014835107 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f0c70005c20 con 0x7f0c8c1048a0 2024-03-21T12:46:47.359 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.754+0000 7f0c7f7fe700 1 -- 172.21.15.43:0/3014835107 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (unknown 2005831594 0 1748205903) 0x7f0c8801e2a0 con 0x7f0c8c1048a0 2024-03-21T12:46:47.359 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.756+0000 7f0c9447c700 1 -- 172.21.15.43:0/3014835107 >> v1:172.21.15.43:6800/1388905519 conn(0x7f0c7803a750 legacy=0x7f0c7803cc10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T12:46:47.360 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.756+0000 7f0c9447c700 1 -- 172.21.15.43:0/3014835107 >> v1:172.21.15.43:6789/0 conn(0x7f0c8c1048a0 legacy=0x7f0c8c19bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:47.360 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.757+0000 7f0c9447c700 1 -- 172.21.15.43:0/3014835107 shutdown_connections 2024-03-21T12:46:47.360 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.757+0000 7f0c9447c700 1 -- 172.21.15.43:0/3014835107 >> 172.21.15.43:0/3014835107 conn(0x7f0c8c1004c0 msgr2=0x7f0c8c192a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:47.360 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.757+0000 7f0c9447c700 1 -- 172.21.15.43:0/3014835107 shutdown_connections 2024-03-21T12:46:47.360 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:46.757+0000 7f0c9447c700 1 -- 172.21.15.43:0/3014835107 wait complete. 2024-03-21T12:46:47.360 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for the mgr to restart... 2024-03-21T12:46:47.361 INFO:teuthology.orchestra.run.smithi043.stdout:Waiting for mgr epoch 9... 2024-03-21T12:46:47.595 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:47 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:47.347+0000 7efd4d04e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T12:46:48.733 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:48.352+0000 7efd4d04e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T12:46:48.733 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:48.487+0000 7efd4d04e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T12:46:49.126 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:48.828+0000 7efd4d04e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T12:46:49.126 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:48.921+0000 7efd4d04e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T12:46:49.126 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:49 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:49.125+0000 7efd4d04e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T12:46:49.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:49 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:49.316+0000 7efd4d04e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T12:46:49.877 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:49 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:49.614+0000 7efd4d04e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T12:46:49.877 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:49 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:49.876+0000 7efd4d04e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T12:46:50.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:50 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:50.123+0000 7efd4d04e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T12:46:51.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:50 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:50.960+0000 7efd4d04e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T12:46:51.871 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:51 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:51.544+0000 7efd4d04e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T12:46:51.872 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:51 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:51.645+0000 7efd4d04e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T12:46:51.872 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:46:51 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:46:51.870+0000 7efd4d04e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T12:46:52.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:51 smithi043 ceph-mon[30062]: Active manager daemon y restarted 2024-03-21T12:46:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:51 smithi043 ceph-mon[30062]: Activating manager daemon y 2024-03-21T12:46:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:51 smithi043 ceph-mon[30062]: osdmap e3: 0 total, 0 up, 0 in 2024-03-21T12:46:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:51 smithi043 ceph-mon[30062]: mgrmap e10: y(active, starting, since 0.00319032s) 2024-03-21T12:46:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:51 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T12:46:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:51 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T12:46:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:51 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T12:46:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:51 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T12:46:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:51 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T12:46:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:51 smithi043 ceph-mon[30062]: Manager daemon y is now available 2024-03-21T12:46:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:51 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:46:53.191 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:52 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-21T12:46:53.191 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:52 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-21T12:46:53.191 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:52 smithi043 ceph-mon[30062]: mgrmap e11: y(active, since 1.00737s) 2024-03-21T12:46:53.800 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout { 2024-03-21T12:46:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-21T12:46:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-21T12:46:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout } 2024-03-21T12:46:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.834+0000 7f23327ae700 1 Processor -- start 2024-03-21T12:46:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.834+0000 7f23327ae700 1 -- start start 2024-03-21T12:46:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.834+0000 7f23327ae700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f232c109320 con 0x7f232c104e70 2024-03-21T12:46:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.834+0000 7f232bfff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f232c104e70 0x7f232c105290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38312/0 (socket says 172.21.15.43:38312) 2024-03-21T12:46:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.834+0000 7f232bfff700 1 -- 172.21.15.43:0/3524026685 learned_addr learned my addr 172.21.15.43:0/3524026685 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.834+0000 7f232affd700 1 -- 172.21.15.43:0/3524026685 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3113822676 0 0) 0x7f232c109320 con 0x7f232c104e70 2024-03-21T12:46:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.834+0000 7f232affd700 1 -- 172.21.15.43:0/3524026685 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23140035b0 con 0x7f232c104e70 2024-03-21T12:46:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.835+0000 7f232affd700 1 -- 172.21.15.43:0/3524026685 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3359291723 0 0) 0x7f23140035b0 con 0x7f232c104e70 2024-03-21T12:46:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.835+0000 7f232affd700 1 -- 172.21.15.43:0/3524026685 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232c10a500 con 0x7f232c104e70 2024-03-21T12:46:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.835+0000 7f232affd700 1 -- 172.21.15.43:0/3524026685 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f231c001580 con 0x7f232c104e70 2024-03-21T12:46:53.801 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.835+0000 7f232affd700 1 -- 172.21.15.43:0/3524026685 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f231c004930 con 0x7f232c104e70 2024-03-21T12:46:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.835+0000 7f232affd700 1 -- 172.21.15.43:0/3524026685 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f231c005a70 con 0x7f232c104e70 2024-03-21T12:46:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.835+0000 7f23327ae700 1 -- 172.21.15.43:0/3524026685 >> v1:172.21.15.43:6789/0 conn(0x7f232c104e70 legacy=0x7f232c105290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.835+0000 7f23327ae700 1 -- 172.21.15.43:0/3524026685 shutdown_connections 2024-03-21T12:46:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.835+0000 7f23327ae700 1 -- 172.21.15.43:0/3524026685 >> 172.21.15.43:0/3524026685 conn(0x7f232c1004c0 msgr2=0x7f232c1028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.836+0000 7f23327ae700 1 -- 172.21.15.43:0/3524026685 shutdown_connections 2024-03-21T12:46:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.836+0000 7f23327ae700 1 -- 172.21.15.43:0/3524026685 wait complete. 2024-03-21T12:46:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.836+0000 7f23327ae700 1 Processor -- start 2024-03-21T12:46:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.836+0000 7f23327ae700 1 -- start start 2024-03-21T12:46:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.836+0000 7f23327ae700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f232c19fae0 con 0x7f232c104e70 2024-03-21T12:46:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.837+0000 7f232bfff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f232c104e70 0x7f232c102330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38320/0 (socket says 172.21.15.43:38320) 2024-03-21T12:46:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.837+0000 7f232bfff700 1 -- 172.21.15.43:0/3122618641 learned_addr learned my addr 172.21.15.43:0/3122618641 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.837+0000 7f23297fa700 1 -- 172.21.15.43:0/3122618641 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2414140778 0 0) 0x7f232c19fae0 con 0x7f232c104e70 2024-03-21T12:46:53.802 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.837+0000 7f23297fa700 1 -- 172.21.15.43:0/3122618641 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23140055f0 con 0x7f232c104e70 2024-03-21T12:46:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.837+0000 7f23297fa700 1 -- 172.21.15.43:0/3122618641 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3074512778 0 0) 0x7f23140055f0 con 0x7f232c104e70 2024-03-21T12:46:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.837+0000 7f23297fa700 1 -- 172.21.15.43:0/3122618641 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f232c19fae0 con 0x7f232c104e70 2024-03-21T12:46:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.837+0000 7f23297fa700 1 -- 172.21.15.43:0/3122618641 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f231c001080 con 0x7f232c104e70 2024-03-21T12:46:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.837+0000 7f23297fa700 1 -- 172.21.15.43:0/3122618641 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 211400734 0 0) 0x7f232c19fae0 con 0x7f232c104e70 2024-03-21T12:46:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.837+0000 7f23297fa700 1 -- 172.21.15.43:0/3122618641 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232c1a0cc0 con 0x7f232c104e70 2024-03-21T12:46:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.837+0000 7f23327ae700 1 -- 172.21.15.43:0/3122618641 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f232c19fd10 con 0x7f232c104e70 2024-03-21T12:46:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.837+0000 7f23327ae700 1 -- 172.21.15.43:0/3122618641 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f232c1a0250 con 0x7f232c104e70 2024-03-21T12:46:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.837+0000 7f23297fa700 1 -- 172.21.15.43:0/3122618641 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f231c0059c0 con 0x7f232c104e70 2024-03-21T12:46:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.837+0000 7f23297fa700 1 -- 172.21.15.43:0/3122618641 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f231c0062f0 con 0x7f232c104e70 2024-03-21T12:46:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.838+0000 7f23297fa700 1 -- 172.21.15.43:0/3122618641 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 9) v1 ==== 46272+0+0 (unknown 1750918330 0 0) 0x7f231c011a60 con 0x7f232c104e70 2024-03-21T12:46:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.838+0000 7f23327ae700 1 -- 172.21.15.43:0/3122618641 --> v1:172.21.15.43:6800/1388905519 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f232c0637c0 con 0x7f231403a650 2024-03-21T12:46:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.838+0000 7f23297fa700 1 -- 172.21.15.43:0/3122618641 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 897+0+0 (unknown 73304150 0 0) 0x7f231c005110 con 0x7f232c104e70 2024-03-21T12:46:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:47.838+0000 7f232b7fe700 1 -- 172.21.15.43:0/3122618641 >> v1:172.21.15.43:6800/1388905519 conn(0x7f231403a650 legacy=0x7f231403cb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.43:6800/1388905519 2024-03-21T12:46:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:48.039+0000 7f232b7fe700 1 -- 172.21.15.43:0/3122618641 >> v1:172.21.15.43:6800/1388905519 conn(0x7f231403a650 legacy=0x7f231403cb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.43:6800/1388905519 2024-03-21T12:46:53.803 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:48.440+0000 7f232b7fe700 1 -- 172.21.15.43:0/3122618641 >> v1:172.21.15.43:6800/1388905519 conn(0x7f231403a650 legacy=0x7f231403cb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.43:6800/1388905519 2024-03-21T12:46:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:49.241+0000 7f232b7fe700 1 -- 172.21.15.43:0/3122618641 >> v1:172.21.15.43:6800/1388905519 conn(0x7f231403a650 legacy=0x7f231403cb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.43:6800/1388905519 2024-03-21T12:46:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:50.842+0000 7f232b7fe700 1 -- 172.21.15.43:0/3122618641 >> v1:172.21.15.43:6800/1388905519 conn(0x7f231403a650 legacy=0x7f231403cb10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.43:6800/1388905519 2024-03-21T12:46:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:51.886+0000 7f23297fa700 1 -- 172.21.15.43:0/3122618641 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mgrmap(e 10) v1 ==== 46074+0+0 (unknown 3028373113 0 0) 0x7f231c013020 con 0x7f232c104e70 2024-03-21T12:46:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:51.886+0000 7f23297fa700 1 -- 172.21.15.43:0/3122618641 >> v1:172.21.15.43:6800/1388905519 conn(0x7f231403a650 legacy=0x7f231403cb10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T12:46:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:52.888+0000 7f23297fa700 1 -- 172.21.15.43:0/3122618641 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mgrmap(e 11) v1 ==== 46166+0+0 (unknown 2502352597 0 0) 0x7f231c048f40 con 0x7f232c104e70 2024-03-21T12:46:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:52.889+0000 7f23297fa700 1 -- 172.21.15.43:0/3122618641 --> v1:172.21.15.43:6800/4081968544 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f231c004930 con 0x7f231403efc0 2024-03-21T12:46:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:52.890+0000 7f23297fa700 1 -- 172.21.15.43:0/3122618641 <== mgr.14152 v1:172.21.15.43:6800/4081968544 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (unknown 0 0 4109177037) 0x7f231c004930 con 0x7f231403efc0 2024-03-21T12:46:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:52.897+0000 7f23327ae700 1 -- 172.21.15.43:0/3122618641 --> v1:172.21.15.43:6800/4081968544 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f232c10a0b0 con 0x7f231403efc0 2024-03-21T12:46:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:52.897+0000 7f23297fa700 1 -- 172.21.15.43:0/3122618641 <== mgr.14152 v1:172.21.15.43:6800/4081968544 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (unknown 0 0 3086460295) 0x7f232c10a0b0 con 0x7f231403efc0 2024-03-21T12:46:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:52.897+0000 7f23327ae700 1 -- 172.21.15.43:0/3122618641 >> v1:172.21.15.43:6800/4081968544 conn(0x7f231403efc0 legacy=0x7f23140413b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:52.897+0000 7f23327ae700 1 -- 172.21.15.43:0/3122618641 >> v1:172.21.15.43:6789/0 conn(0x7f232c104e70 legacy=0x7f232c102330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:52.897+0000 7f23327ae700 1 -- 172.21.15.43:0/3122618641 shutdown_connections 2024-03-21T12:46:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:52.897+0000 7f23327ae700 1 -- 172.21.15.43:0/3122618641 >> 172.21.15.43:0/3122618641 conn(0x7f232c1004c0 msgr2=0x7f232c1083b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:53.804 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:52.897+0000 7f23327ae700 1 -- 172.21.15.43:0/3122618641 shutdown_connections 2024-03-21T12:46:53.805 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:52.897+0000 7f23327ae700 1 -- 172.21.15.43:0/3122618641 wait complete. 2024-03-21T12:46:53.805 INFO:teuthology.orchestra.run.smithi043.stdout:mgr epoch 9 is available 2024-03-21T12:46:53.805 INFO:teuthology.orchestra.run.smithi043.stdout:Generating a dashboard self-signed certificate... 2024-03-21T12:46:54.856 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:54 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:46:54.856 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:54 smithi043 ceph-mon[30062]: [21/Mar/2024:12:46:53] ENGINE Bus STARTING 2024-03-21T12:46:55.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-21T12:46:55.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.275+0000 7f66e4faf700 1 Processor -- start 2024-03-21T12:46:55.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.275+0000 7f66e4faf700 1 -- start start 2024-03-21T12:46:55.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.276+0000 7f66e4faf700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f66e010a420 con 0x7f66e0105ff0 2024-03-21T12:46:55.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.276+0000 7f66de59c700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f66e0105ff0 0x7f66e0106410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57838/0 (socket says 172.21.15.43:57838) 2024-03-21T12:46:55.138 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.276+0000 7f66de59c700 1 -- 172.21.15.43:0/4146770538 learned_addr learned my addr 172.21.15.43:0/4146770538 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:55.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.276+0000 7f66dd59a700 1 -- 172.21.15.43:0/4146770538 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3569863379 0 0) 0x7f66e010a420 con 0x7f66e0105ff0 2024-03-21T12:46:55.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.276+0000 7f66dd59a700 1 -- 172.21.15.43:0/4146770538 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66c80035b0 con 0x7f66e0105ff0 2024-03-21T12:46:55.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.276+0000 7f66dd59a700 1 -- 172.21.15.43:0/4146770538 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1375473111 0 0) 0x7f66c80035b0 con 0x7f66e0105ff0 2024-03-21T12:46:55.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.277+0000 7f66dd59a700 1 -- 172.21.15.43:0/4146770538 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e010b600 con 0x7f66e0105ff0 2024-03-21T12:46:55.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.277+0000 7f66dd59a700 1 -- 172.21.15.43:0/4146770538 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f66d4001580 con 0x7f66e0105ff0 2024-03-21T12:46:55.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.277+0000 7f66dd59a700 1 -- 172.21.15.43:0/4146770538 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f66d4001c70 con 0x7f66e0105ff0 2024-03-21T12:46:55.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.277+0000 7f66dd59a700 1 -- 172.21.15.43:0/4146770538 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f66d4005960 con 0x7f66e0105ff0 2024-03-21T12:46:55.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.277+0000 7f66e4faf700 1 -- 172.21.15.43:0/4146770538 >> v1:172.21.15.43:6789/0 conn(0x7f66e0105ff0 legacy=0x7f66e0106410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:55.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.277+0000 7f66e4faf700 1 -- 172.21.15.43:0/4146770538 shutdown_connections 2024-03-21T12:46:55.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.277+0000 7f66e4faf700 1 -- 172.21.15.43:0/4146770538 >> 172.21.15.43:0/4146770538 conn(0x7f66e01015f0 msgr2=0x7f66e0103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:55.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.277+0000 7f66e4faf700 1 -- 172.21.15.43:0/4146770538 shutdown_connections 2024-03-21T12:46:55.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.277+0000 7f66e4faf700 1 -- 172.21.15.43:0/4146770538 wait complete. 2024-03-21T12:46:55.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.278+0000 7f66e4faf700 1 Processor -- start 2024-03-21T12:46:55.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.278+0000 7f66e4faf700 1 -- start start 2024-03-21T12:46:55.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.278+0000 7f66e4faf700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f66e019fb50 con 0x7f66e0105ff0 2024-03-21T12:46:55.139 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.278+0000 7f66de59c700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f66e0105ff0 0x7f66e007b410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57854/0 (socket says 172.21.15.43:57854) 2024-03-21T12:46:55.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.278+0000 7f66de59c700 1 -- 172.21.15.43:0/1982450888 learned_addr learned my addr 172.21.15.43:0/1982450888 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:55.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.279+0000 7f66cf7fe700 1 -- 172.21.15.43:0/1982450888 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 756109207 0 0) 0x7f66e019fb50 con 0x7f66e0105ff0 2024-03-21T12:46:55.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.279+0000 7f66cf7fe700 1 -- 172.21.15.43:0/1982450888 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66c8005640 con 0x7f66e0105ff0 2024-03-21T12:46:55.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.279+0000 7f66cf7fe700 1 -- 172.21.15.43:0/1982450888 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3008063812 0 0) 0x7f66c8005640 con 0x7f66e0105ff0 2024-03-21T12:46:55.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.279+0000 7f66cf7fe700 1 -- 172.21.15.43:0/1982450888 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f66e019fb50 con 0x7f66e0105ff0 2024-03-21T12:46:55.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.279+0000 7f66cf7fe700 1 -- 172.21.15.43:0/1982450888 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f66d4001080 con 0x7f66e0105ff0 2024-03-21T12:46:55.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.279+0000 7f66cf7fe700 1 -- 172.21.15.43:0/1982450888 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1632970097 0 0) 0x7f66e019fb50 con 0x7f66e0105ff0 2024-03-21T12:46:55.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.279+0000 7f66cf7fe700 1 -- 172.21.15.43:0/1982450888 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e01a0d30 con 0x7f66e0105ff0 2024-03-21T12:46:55.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.279+0000 7f66e4faf700 1 -- 172.21.15.43:0/1982450888 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e019fd80 con 0x7f66e0105ff0 2024-03-21T12:46:55.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.279+0000 7f66e4faf700 1 -- 172.21.15.43:0/1982450888 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e01a02c0 con 0x7f66e0105ff0 2024-03-21T12:46:55.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.279+0000 7f66cf7fe700 1 -- 172.21.15.43:0/1982450888 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f66d4005620 con 0x7f66e0105ff0 2024-03-21T12:46:55.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.279+0000 7f66cf7fe700 1 -- 172.21.15.43:0/1982450888 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f66d4005fd0 con 0x7f66e0105ff0 2024-03-21T12:46:55.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.280+0000 7f66cf7fe700 1 -- 172.21.15.43:0/1982450888 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 11) v1 ==== 46166+0+0 (unknown 2502352597 0 0) 0x7f66d40115d0 con 0x7f66e0105ff0 2024-03-21T12:46:55.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.280+0000 7f66e4faf700 1 -- 172.21.15.43:0/1982450888 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c0005150 con 0x7f66e0105ff0 2024-03-21T12:46:55.140 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.281+0000 7f66cf7fe700 1 -- 172.21.15.43:0/1982450888 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1006335234 0 0) 0x7f66d40478d0 con 0x7f66e0105ff0 2024-03-21T12:46:55.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.285+0000 7f66cf7fe700 1 -- 172.21.15.43:0/1982450888 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f66d4017550 con 0x7f66e0105ff0 2024-03-21T12:46:55.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.500+0000 7f66e4faf700 1 -- 172.21.15.43:0/1982450888 --> v1:172.21.15.43:6800/4081968544 -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f66c0000bc0 con 0x7f66c803a5c0 2024-03-21T12:46:55.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.503+0000 7f66cf7fe700 1 -- 172.21.15.43:0/1982450888 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mgrmap(e 12) v1 ==== 46272+0+0 (unknown 2237625176 0 0) 0x7f66d4012b90 con 0x7f66e0105ff0 2024-03-21T12:46:55.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.549+0000 7f66cf7fe700 1 -- 172.21.15.43:0/1982450888 <== mgr.14152 v1:172.21.15.43:6800/4081968544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (unknown 0 0 3317752739) 0x7f66c0000bc0 con 0x7f66c803a5c0 2024-03-21T12:46:55.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.551+0000 7f66e4faf700 1 -- 172.21.15.43:0/1982450888 >> v1:172.21.15.43:6800/4081968544 conn(0x7f66c803a5c0 legacy=0x7f66c803ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:55.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.551+0000 7f66e4faf700 1 -- 172.21.15.43:0/1982450888 >> v1:172.21.15.43:6789/0 conn(0x7f66e0105ff0 legacy=0x7f66e007b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:55.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.551+0000 7f66e4faf700 1 -- 172.21.15.43:0/1982450888 shutdown_connections 2024-03-21T12:46:55.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.551+0000 7f66e4faf700 1 -- 172.21.15.43:0/1982450888 >> 172.21.15.43:0/1982450888 conn(0x7f66e01015f0 msgr2=0x7f66e01039e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:55.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.551+0000 7f66e4faf700 1 -- 172.21.15.43:0/1982450888 shutdown_connections 2024-03-21T12:46:55.141 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:54.551+0000 7f66e4faf700 1 -- 172.21.15.43:0/1982450888 wait complete. 2024-03-21T12:46:55.141 INFO:teuthology.orchestra.run.smithi043.stdout:Creating initial admin user... 2024-03-21T12:46:55.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:55 smithi043 ceph-mon[30062]: [21/Mar/2024:12:46:53] ENGINE Serving on https://172.21.15.43:7150 2024-03-21T12:46:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:55 smithi043 ceph-mon[30062]: [21/Mar/2024:12:46:54] ENGINE Serving on http://172.21.15.43:8765 2024-03-21T12:46:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:55 smithi043 ceph-mon[30062]: [21/Mar/2024:12:46:54] ENGINE Bus STARTED 2024-03-21T12:46:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:55 smithi043 ceph-mon[30062]: from='client.14164 v1:172.21.15.43:0/1982450888' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:46:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:55 smithi043 ceph-mon[30062]: mgrmap e12: y(active, since 2s) 2024-03-21T12:46:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:55 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:46:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:55 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:46:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:55 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:46:57.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:57 smithi043 ceph-mon[30062]: from='client.14166 v1:172.21.15.43:0/4200300975' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:46:57.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:57 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:46:57.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Jsf26po.SGRurKmIx1KcLODPbiDUvM6FCenOOmLgAxprn1pGCApXC", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711025216, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-21T12:46:57.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.635+0000 7f78deee7700 1 Processor -- start 2024-03-21T12:46:57.396 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.635+0000 7f78deee7700 1 -- start start 2024-03-21T12:46:57.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.635+0000 7f78deee7700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f78d81092e0 con 0x7f78d8106490 2024-03-21T12:46:57.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.636+0000 7f78dcc83700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f78d8106490 0x7f78d81068b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57862/0 (socket says 172.21.15.43:57862) 2024-03-21T12:46:57.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.636+0000 7f78dcc83700 1 -- 172.21.15.43:0/2005080811 learned_addr learned my addr 172.21.15.43:0/2005080811 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:57.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.636+0000 7f78d77fe700 1 -- 172.21.15.43:0/2005080811 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968411165 0 0) 0x7f78d81092e0 con 0x7f78d8106490 2024-03-21T12:46:57.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.636+0000 7f78d77fe700 1 -- 172.21.15.43:0/2005080811 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78c80057c0 con 0x7f78d8106490 2024-03-21T12:46:57.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.636+0000 7f78d77fe700 1 -- 172.21.15.43:0/2005080811 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3608798787 0 0) 0x7f78c80057c0 con 0x7f78d8106490 2024-03-21T12:46:57.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.636+0000 7f78d77fe700 1 -- 172.21.15.43:0/2005080811 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d810a4c0 con 0x7f78d8106490 2024-03-21T12:46:57.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.636+0000 7f78d77fe700 1 -- 172.21.15.43:0/2005080811 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f78cc001580 con 0x7f78d8106490 2024-03-21T12:46:57.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.636+0000 7f78d77fe700 1 -- 172.21.15.43:0/2005080811 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f78cc001c70 con 0x7f78d8106490 2024-03-21T12:46:57.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.636+0000 7f78d77fe700 1 -- 172.21.15.43:0/2005080811 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f78cc005960 con 0x7f78d8106490 2024-03-21T12:46:57.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.637+0000 7f78deee7700 1 -- 172.21.15.43:0/2005080811 >> v1:172.21.15.43:6789/0 conn(0x7f78d8106490 legacy=0x7f78d81068b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:57.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.637+0000 7f78deee7700 1 -- 172.21.15.43:0/2005080811 shutdown_connections 2024-03-21T12:46:57.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.637+0000 7f78deee7700 1 -- 172.21.15.43:0/2005080811 >> 172.21.15.43:0/2005080811 conn(0x7f78d80ffcb0 msgr2=0x7f78d81020d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:57.397 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.637+0000 7f78deee7700 1 -- 172.21.15.43:0/2005080811 shutdown_connections 2024-03-21T12:46:57.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.637+0000 7f78deee7700 1 -- 172.21.15.43:0/2005080811 wait complete. 2024-03-21T12:46:57.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.638+0000 7f78deee7700 1 Processor -- start 2024-03-21T12:46:57.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.638+0000 7f78deee7700 1 -- start start 2024-03-21T12:46:57.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.638+0000 7f78deee7700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f78d819c8e0 con 0x7f78d819bdb0 2024-03-21T12:46:57.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.638+0000 7f78dcc83700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f78d819bdb0 0x7f78d819c1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57878/0 (socket says 172.21.15.43:57878) 2024-03-21T12:46:57.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.638+0000 7f78dcc83700 1 -- 172.21.15.43:0/4200300975 learned_addr learned my addr 172.21.15.43:0/4200300975 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:57.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.638+0000 7f78d5ffb700 1 -- 172.21.15.43:0/4200300975 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4205686070 0 0) 0x7f78d819c8e0 con 0x7f78d819bdb0 2024-03-21T12:46:57.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.638+0000 7f78d5ffb700 1 -- 172.21.15.43:0/4200300975 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78c8004730 con 0x7f78d819bdb0 2024-03-21T12:46:57.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.638+0000 7f78d5ffb700 1 -- 172.21.15.43:0/4200300975 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2242915947 0 0) 0x7f78c8004730 con 0x7f78d819bdb0 2024-03-21T12:46:57.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.638+0000 7f78d5ffb700 1 -- 172.21.15.43:0/4200300975 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f78d819c8e0 con 0x7f78d819bdb0 2024-03-21T12:46:57.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.638+0000 7f78d5ffb700 1 -- 172.21.15.43:0/4200300975 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f78cc001080 con 0x7f78d819bdb0 2024-03-21T12:46:57.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.639+0000 7f78d5ffb700 1 -- 172.21.15.43:0/4200300975 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 253059115 0 0) 0x7f78d819c8e0 con 0x7f78d819bdb0 2024-03-21T12:46:57.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.639+0000 7f78d5ffb700 1 -- 172.21.15.43:0/4200300975 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d81a12d0 con 0x7f78d819bdb0 2024-03-21T12:46:57.398 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.639+0000 7f78deee7700 1 -- 172.21.15.43:0/4200300975 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d81a02c0 con 0x7f78d819bdb0 2024-03-21T12:46:57.399 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.639+0000 7f78deee7700 1 -- 172.21.15.43:0/4200300975 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d81a0800 con 0x7f78d819bdb0 2024-03-21T12:46:57.399 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.639+0000 7f78d5ffb700 1 -- 172.21.15.43:0/4200300975 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f78cc005600 con 0x7f78d819bdb0 2024-03-21T12:46:57.399 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.639+0000 7f78d5ffb700 1 -- 172.21.15.43:0/4200300975 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f78cc0061c0 con 0x7f78d819bdb0 2024-03-21T12:46:57.399 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.640+0000 7f78d5ffb700 1 -- 172.21.15.43:0/4200300975 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 12) v1 ==== 46272+0+0 (unknown 2237625176 0 0) 0x7f78cc007490 con 0x7f78d819bdb0 2024-03-21T12:46:57.399 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.640+0000 7f78d5ffb700 1 -- 172.21.15.43:0/4200300975 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1006335234 0 0) 0x7f78cc0488c0 con 0x7f78d819bdb0 2024-03-21T12:46:57.399 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.640+0000 7f78deee7700 1 -- 172.21.15.43:0/4200300975 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78bc005150 con 0x7f78d819bdb0 2024-03-21T12:46:57.399 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.645+0000 7f78d5ffb700 1 -- 172.21.15.43:0/4200300975 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f78cc0174e0 con 0x7f78d819bdb0 2024-03-21T12:46:57.399 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:55.880+0000 7f78deee7700 1 -- 172.21.15.43:0/4200300975 --> v1:172.21.15.43:6800/4081968544 -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f78bc002430 con 0x7f78c803a750 2024-03-21T12:46:57.399 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:56.127+0000 7f78d5ffb700 1 -- 172.21.15.43:0/4200300975 <== mgr.14152 v1:172.21.15.43:6800/4081968544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (unknown 0 0 3686166849) 0x7f78bc002430 con 0x7f78c803a750 2024-03-21T12:46:57.399 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:56.129+0000 7f78deee7700 1 -- 172.21.15.43:0/4200300975 >> v1:172.21.15.43:6800/4081968544 conn(0x7f78c803a750 legacy=0x7f78c803cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:57.399 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:56.129+0000 7f78deee7700 1 -- 172.21.15.43:0/4200300975 >> v1:172.21.15.43:6789/0 conn(0x7f78d819bdb0 legacy=0x7f78d819c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:57.399 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:56.129+0000 7f78deee7700 1 -- 172.21.15.43:0/4200300975 shutdown_connections 2024-03-21T12:46:57.399 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:56.129+0000 7f78deee7700 1 -- 172.21.15.43:0/4200300975 >> 172.21.15.43:0/4200300975 conn(0x7f78d80ffcb0 msgr2=0x7f78d81020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:57.399 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:56.129+0000 7f78deee7700 1 -- 172.21.15.43:0/4200300975 shutdown_connections 2024-03-21T12:46:57.399 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:56.129+0000 7f78deee7700 1 -- 172.21.15.43:0/4200300975 wait complete. 2024-03-21T12:46:57.399 INFO:teuthology.orchestra.run.smithi043.stdout:Fetching dashboard port number... 2024-03-21T12:46:58.360 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:58 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2794313369' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-21T12:46:58.644 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stdout 8443 2024-03-21T12:46:58.644 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.794+0000 7f25572f0700 1 Processor -- start 2024-03-21T12:46:58.644 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.794+0000 7f25572f0700 1 -- start start 2024-03-21T12:46:58.644 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.795+0000 7f25572f0700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2550079fe0 con 0x7f2550106a30 2024-03-21T12:46:58.644 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.795+0000 7f255508c700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2550106a30 0x7f2550108e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57892/0 (socket says 172.21.15.43:57892) 2024-03-21T12:46:58.644 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.795+0000 7f255508c700 1 -- 172.21.15.43:0/540878874 learned_addr learned my addr 172.21.15.43:0/540878874 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:58.644 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.795+0000 7f254ffff700 1 -- 172.21.15.43:0/540878874 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1630070489 0 0) 0x7f2550079fe0 con 0x7f2550106a30 2024-03-21T12:46:58.644 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.795+0000 7f254ffff700 1 -- 172.21.15.43:0/540878874 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25380035b0 con 0x7f2550106a30 2024-03-21T12:46:58.644 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.795+0000 7f254ffff700 1 -- 172.21.15.43:0/540878874 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1090462525 0 0) 0x7f25380035b0 con 0x7f2550106a30 2024-03-21T12:46:58.645 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.795+0000 7f254ffff700 1 -- 172.21.15.43:0/540878874 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f255010b550 con 0x7f2550106a30 2024-03-21T12:46:58.645 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.795+0000 7f254ffff700 1 -- 172.21.15.43:0/540878874 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f2548001580 con 0x7f2550106a30 2024-03-21T12:46:58.645 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.795+0000 7f254ffff700 1 -- 172.21.15.43:0/540878874 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f2548004930 con 0x7f2550106a30 2024-03-21T12:46:58.645 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.795+0000 7f254ffff700 1 -- 172.21.15.43:0/540878874 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f2548005a70 con 0x7f2550106a30 2024-03-21T12:46:58.645 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.796+0000 7f25572f0700 1 -- 172.21.15.43:0/540878874 >> v1:172.21.15.43:6789/0 conn(0x7f2550106a30 legacy=0x7f2550108e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:58.645 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.796+0000 7f25572f0700 1 -- 172.21.15.43:0/540878874 shutdown_connections 2024-03-21T12:46:58.645 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.796+0000 7f25572f0700 1 -- 172.21.15.43:0/540878874 >> 172.21.15.43:0/540878874 conn(0x7f2550100460 msgr2=0x7f25501028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:58.645 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.796+0000 7f25572f0700 1 -- 172.21.15.43:0/540878874 shutdown_connections 2024-03-21T12:46:58.645 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.796+0000 7f25572f0700 1 -- 172.21.15.43:0/540878874 wait complete. 2024-03-21T12:46:58.645 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.797+0000 7f25572f0700 1 Processor -- start 2024-03-21T12:46:58.645 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.797+0000 7f25572f0700 1 -- start start 2024-03-21T12:46:58.645 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.797+0000 7f25572f0700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f255019fb40 con 0x7f2550106a30 2024-03-21T12:46:58.645 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.797+0000 7f255508c700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2550106a30 0x7f25501062b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57900/0 (socket says 172.21.15.43:57900) 2024-03-21T12:46:58.645 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.797+0000 7f255508c700 1 -- 172.21.15.43:0/2794313369 learned_addr learned my addr 172.21.15.43:0/2794313369 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:58.645 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.797+0000 7f254e7fc700 1 -- 172.21.15.43:0/2794313369 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2085275240 0 0) 0x7f255019fb40 con 0x7f2550106a30 2024-03-21T12:46:58.646 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.797+0000 7f254e7fc700 1 -- 172.21.15.43:0/2794313369 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25380055f0 con 0x7f2550106a30 2024-03-21T12:46:58.646 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.797+0000 7f254e7fc700 1 -- 172.21.15.43:0/2794313369 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 253189042 0 0) 0x7f25380055f0 con 0x7f2550106a30 2024-03-21T12:46:58.646 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.797+0000 7f254e7fc700 1 -- 172.21.15.43:0/2794313369 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f255019fb40 con 0x7f2550106a30 2024-03-21T12:46:58.646 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.797+0000 7f254e7fc700 1 -- 172.21.15.43:0/2794313369 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f2548001080 con 0x7f2550106a30 2024-03-21T12:46:58.646 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.797+0000 7f254e7fc700 1 -- 172.21.15.43:0/2794313369 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4115087986 0 0) 0x7f255019fb40 con 0x7f2550106a30 2024-03-21T12:46:58.646 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.797+0000 7f254e7fc700 1 -- 172.21.15.43:0/2794313369 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25501a0d20 con 0x7f2550106a30 2024-03-21T12:46:58.646 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.797+0000 7f25572f0700 1 -- 172.21.15.43:0/2794313369 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f255019fd70 con 0x7f2550106a30 2024-03-21T12:46:58.646 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.798+0000 7f25572f0700 1 -- 172.21.15.43:0/2794313369 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f25501a02b0 con 0x7f2550106a30 2024-03-21T12:46:58.646 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.798+0000 7f254e7fc700 1 -- 172.21.15.43:0/2794313369 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f2548005980 con 0x7f2550106a30 2024-03-21T12:46:58.646 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.798+0000 7f254e7fc700 1 -- 172.21.15.43:0/2794313369 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f25480062f0 con 0x7f2550106a30 2024-03-21T12:46:58.646 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.798+0000 7f254e7fc700 1 -- 172.21.15.43:0/2794313369 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 12) v1 ==== 46272+0+0 (unknown 2237625176 0 0) 0x7f2548007580 con 0x7f2550106a30 2024-03-21T12:46:58.646 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.798+0000 7f25572f0700 1 -- 172.21.15.43:0/2794313369 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25500637c0 con 0x7f2550106a30 2024-03-21T12:46:58.646 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.799+0000 7f254e7fc700 1 -- 172.21.15.43:0/2794313369 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1006335234 0 0) 0x7f2548047ad0 con 0x7f2550106a30 2024-03-21T12:46:58.646 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:57.803+0000 7f254e7fc700 1 -- 172.21.15.43:0/2794313369 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f2548017600 con 0x7f2550106a30 2024-03-21T12:46:58.647 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:58.008+0000 7f25572f0700 1 -- 172.21.15.43:0/2794313369 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f255002da30 con 0x7f2550106a30 2024-03-21T12:46:58.647 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:58.008+0000 7f254e7fc700 1 -- 172.21.15.43:0/2794313369 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (unknown 3713421687 0 83753974) 0x7f2548016f00 con 0x7f2550106a30 2024-03-21T12:46:58.647 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:58.010+0000 7f25572f0700 1 -- 172.21.15.43:0/2794313369 >> v1:172.21.15.43:6800/4081968544 conn(0x7f253803a7a0 legacy=0x7f253803cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:58.647 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:58.010+0000 7f25572f0700 1 -- 172.21.15.43:0/2794313369 >> v1:172.21.15.43:6789/0 conn(0x7f2550106a30 legacy=0x7f25501062b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:58.647 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:58.010+0000 7f25572f0700 1 -- 172.21.15.43:0/2794313369 shutdown_connections 2024-03-21T12:46:58.647 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:58.010+0000 7f25572f0700 1 -- 172.21.15.43:0/2794313369 >> 172.21.15.43:0/2794313369 conn(0x7f2550100460 msgr2=0x7f2550190840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:58.647 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:58.010+0000 7f25572f0700 1 -- 172.21.15.43:0/2794313369 shutdown_connections 2024-03-21T12:46:58.647 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:58.010+0000 7f25572f0700 1 -- 172.21.15.43:0/2794313369 wait complete. 2024-03-21T12:46:58.651 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T12:46:58.651 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout disabled 2024-03-21T12:46:58.658 INFO:teuthology.orchestra.run.smithi043.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T12:46:58.658 INFO:teuthology.orchestra.run.smithi043.stdout:systemctl: stdout inactive 2024-03-21T12:46:58.658 INFO:teuthology.orchestra.run.smithi043.stdout:firewalld.service is not enabled 2024-03-21T12:46:58.658 INFO:teuthology.orchestra.run.smithi043.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-21T12:46:58.660 INFO:teuthology.orchestra.run.smithi043.stdout:Ceph Dashboard is now available at: 2024-03-21T12:46:58.660 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:46:58.660 INFO:teuthology.orchestra.run.smithi043.stdout: URL: https://smithi043.front.sepia.ceph.com:8443/ 2024-03-21T12:46:58.660 INFO:teuthology.orchestra.run.smithi043.stdout: User: admin 2024-03-21T12:46:58.660 INFO:teuthology.orchestra.run.smithi043.stdout: Password: wqswi6ryc6 2024-03-21T12:46:58.661 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:46:58.661 INFO:teuthology.orchestra.run.smithi043.stdout:Saving cluster configuration to /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config directory 2024-03-21T12:46:58.661 INFO:teuthology.orchestra.run.smithi043.stdout:Enabling autotune for osd_memory_target 2024-03-21T12:46:59.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:46:59 smithi043 ceph-mon[30062]: mgrmap e13: y(active, since 6s) 2024-03-21T12:46:59.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.120+0000 7fb05f775700 1 Processor -- start 2024-03-21T12:46:59.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.120+0000 7fb05f775700 1 -- start start 2024-03-21T12:46:59.975 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.121+0000 7fb05f775700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb058079f80 con 0x7fb058104880 2024-03-21T12:46:59.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.121+0000 7fb05d511700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb058104880 0x7fb058104ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57912/0 (socket says 172.21.15.43:57912) 2024-03-21T12:46:59.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.121+0000 7fb05d511700 1 -- 172.21.15.43:0/2859600935 learned_addr learned my addr 172.21.15.43:0/2859600935 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:59.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.121+0000 7fb04ffff700 1 -- 172.21.15.43:0/2859600935 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506115666 0 0) 0x7fb058079f80 con 0x7fb058104880 2024-03-21T12:46:59.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.121+0000 7fb04ffff700 1 -- 172.21.15.43:0/2859600935 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0440035b0 con 0x7fb058104880 2024-03-21T12:46:59.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.121+0000 7fb04ffff700 1 -- 172.21.15.43:0/2859600935 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2783665877 0 0) 0x7fb0440035b0 con 0x7fb058104880 2024-03-21T12:46:59.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.121+0000 7fb04ffff700 1 -- 172.21.15.43:0/2859600935 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0581063c0 con 0x7fb058104880 2024-03-21T12:46:59.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.121+0000 7fb04ffff700 1 -- 172.21.15.43:0/2859600935 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fb054001580 con 0x7fb058104880 2024-03-21T12:46:59.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.121+0000 7fb04ffff700 1 -- 172.21.15.43:0/2859600935 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7fb054004930 con 0x7fb058104880 2024-03-21T12:46:59.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.121+0000 7fb04ffff700 1 -- 172.21.15.43:0/2859600935 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fb054005a70 con 0x7fb058104880 2024-03-21T12:46:59.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.122+0000 7fb05f775700 1 -- 172.21.15.43:0/2859600935 >> v1:172.21.15.43:6789/0 conn(0x7fb058104880 legacy=0x7fb058104ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:59.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.122+0000 7fb05f775700 1 -- 172.21.15.43:0/2859600935 shutdown_connections 2024-03-21T12:46:59.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.122+0000 7fb05f775700 1 -- 172.21.15.43:0/2859600935 >> 172.21.15.43:0/2859600935 conn(0x7fb058100460 msgr2=0x7fb0581028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:59.976 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.122+0000 7fb05f775700 1 -- 172.21.15.43:0/2859600935 shutdown_connections 2024-03-21T12:46:59.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.122+0000 7fb05f775700 1 -- 172.21.15.43:0/2859600935 wait complete. 2024-03-21T12:46:59.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.123+0000 7fb05f775700 1 Processor -- start 2024-03-21T12:46:59.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.123+0000 7fb05f775700 1 -- start start 2024-03-21T12:46:59.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.123+0000 7fb05f775700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb05810aa30 con 0x7fb058104880 2024-03-21T12:46:59.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.123+0000 7fb05d511700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb058104880 0x7fb05810a320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57924/0 (socket says 172.21.15.43:57924) 2024-03-21T12:46:59.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.123+0000 7fb05d511700 1 -- 172.21.15.43:0/657691219 learned_addr learned my addr 172.21.15.43:0/657691219 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:46:59.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.123+0000 7fb04e7fc700 1 -- 172.21.15.43:0/657691219 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4747003 0 0) 0x7fb05810aa30 con 0x7fb058104880 2024-03-21T12:46:59.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.123+0000 7fb04e7fc700 1 -- 172.21.15.43:0/657691219 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0440055f0 con 0x7fb058104880 2024-03-21T12:46:59.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.123+0000 7fb04e7fc700 1 -- 172.21.15.43:0/657691219 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2123455357 0 0) 0x7fb0440055f0 con 0x7fb058104880 2024-03-21T12:46:59.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.123+0000 7fb04e7fc700 1 -- 172.21.15.43:0/657691219 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb05810aa30 con 0x7fb058104880 2024-03-21T12:46:59.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.123+0000 7fb04e7fc700 1 -- 172.21.15.43:0/657691219 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fb054001080 con 0x7fb058104880 2024-03-21T12:46:59.977 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.123+0000 7fb04e7fc700 1 -- 172.21.15.43:0/657691219 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3456712996 0 0) 0x7fb05810aa30 con 0x7fb058104880 2024-03-21T12:46:59.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.123+0000 7fb04e7fc700 1 -- 172.21.15.43:0/657691219 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb05810ac00 con 0x7fb058104880 2024-03-21T12:46:59.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.123+0000 7fb05f775700 1 -- 172.21.15.43:0/657691219 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb058106e60 con 0x7fb058104880 2024-03-21T12:46:59.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.123+0000 7fb05f775700 1 -- 172.21.15.43:0/657691219 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0581073a0 con 0x7fb058104880 2024-03-21T12:46:59.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.124+0000 7fb04e7fc700 1 -- 172.21.15.43:0/657691219 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7fb054004930 con 0x7fb058104880 2024-03-21T12:46:59.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.124+0000 7fb04e7fc700 1 -- 172.21.15.43:0/657691219 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fb054005fe0 con 0x7fb058104880 2024-03-21T12:46:59.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.124+0000 7fb04e7fc700 1 -- 172.21.15.43:0/657691219 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 13) v1 ==== 46315+0+0 (unknown 152183610 0 0) 0x7fb054011680 con 0x7fb058104880 2024-03-21T12:46:59.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.124+0000 7fb05f775700 1 -- 172.21.15.43:0/657691219 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03c005150 con 0x7fb058104880 2024-03-21T12:46:59.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.124+0000 7fb04e7fc700 1 -- 172.21.15.43:0/657691219 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1006335234 0 0) 0x7fb054047c30 con 0x7fb058104880 2024-03-21T12:46:59.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.129+0000 7fb04e7fc700 1 -- 172.21.15.43:0/657691219 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7fb0540176c0 con 0x7fb058104880 2024-03-21T12:46:59.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.333+0000 7fb05f775700 1 -- 172.21.15.43:0/657691219 --> v1:172.21.15.43:6789/0 -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fb03c005440 con 0x7fb058104880 2024-03-21T12:46:59.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.333+0000 7fb04e7fc700 1 -- 172.21.15.43:0/657691219 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (unknown 689008156 0 0) 0x7fb054016fc0 con 0x7fb058104880 2024-03-21T12:46:59.978 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.335+0000 7fb05f775700 1 -- 172.21.15.43:0/657691219 >> v1:172.21.15.43:6800/4081968544 conn(0x7fb04403a750 legacy=0x7fb04403cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:59.979 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.335+0000 7fb05f775700 1 -- 172.21.15.43:0/657691219 >> v1:172.21.15.43:6789/0 conn(0x7fb058104880 legacy=0x7fb05810a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:46:59.979 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.335+0000 7fb05f775700 1 -- 172.21.15.43:0/657691219 shutdown_connections 2024-03-21T12:46:59.979 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.335+0000 7fb05f775700 1 -- 172.21.15.43:0/657691219 >> 172.21.15.43:0/657691219 conn(0x7fb058100460 msgr2=0x7fb0581028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:46:59.979 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.336+0000 7fb05f775700 1 -- 172.21.15.43:0/657691219 shutdown_connections 2024-03-21T12:46:59.979 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:46:59.336+0000 7fb05f775700 1 -- 172.21.15.43:0/657691219 wait complete. 2024-03-21T12:47:01.389 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.462+0000 7fb51290f700 1 Processor -- start 2024-03-21T12:47:01.389 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.462+0000 7fb51290f700 1 -- start start 2024-03-21T12:47:01.389 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.462+0000 7fb51290f700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb50c109330 con 0x7fb50c104e80 2024-03-21T12:47:01.389 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.462+0000 7fb50bfff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb50c104e80 0x7fb50c1052a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57928/0 (socket says 172.21.15.43:57928) 2024-03-21T12:47:01.389 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.463+0000 7fb50bfff700 1 -- 172.21.15.43:0/2068071257 learned_addr learned my addr 172.21.15.43:0/2068071257 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:47:01.390 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.463+0000 7fb50affd700 1 -- 172.21.15.43:0/2068071257 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4216610130 0 0) 0x7fb50c109330 con 0x7fb50c104e80 2024-03-21T12:47:01.390 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.463+0000 7fb50affd700 1 -- 172.21.15.43:0/2068071257 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4f40035b0 con 0x7fb50c104e80 2024-03-21T12:47:01.390 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.463+0000 7fb50affd700 1 -- 172.21.15.43:0/2068071257 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1758333188 0 0) 0x7fb4f40035b0 con 0x7fb50c104e80 2024-03-21T12:47:01.390 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.463+0000 7fb50affd700 1 -- 172.21.15.43:0/2068071257 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50c10a510 con 0x7fb50c104e80 2024-03-21T12:47:01.390 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.463+0000 7fb50affd700 1 -- 172.21.15.43:0/2068071257 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fb4fc001580 con 0x7fb50c104e80 2024-03-21T12:47:01.390 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.463+0000 7fb50affd700 1 -- 172.21.15.43:0/2068071257 <== mon.0 v1:172.21.15.43:6789/0 4 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7fb4fc004930 con 0x7fb50c104e80 2024-03-21T12:47:01.390 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.463+0000 7fb50affd700 1 -- 172.21.15.43:0/2068071257 <== mon.0 v1:172.21.15.43:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fb4fc005a70 con 0x7fb50c104e80 2024-03-21T12:47:01.390 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.464+0000 7fb51290f700 1 -- 172.21.15.43:0/2068071257 >> v1:172.21.15.43:6789/0 conn(0x7fb50c104e80 legacy=0x7fb50c1052a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:01.390 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.464+0000 7fb51290f700 1 -- 172.21.15.43:0/2068071257 shutdown_connections 2024-03-21T12:47:01.390 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.464+0000 7fb51290f700 1 -- 172.21.15.43:0/2068071257 >> 172.21.15.43:0/2068071257 conn(0x7fb50c1004f0 msgr2=0x7fb50c102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:47:01.390 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.464+0000 7fb51290f700 1 -- 172.21.15.43:0/2068071257 shutdown_connections 2024-03-21T12:47:01.390 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.464+0000 7fb51290f700 1 -- 172.21.15.43:0/2068071257 wait complete. 2024-03-21T12:47:01.390 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.464+0000 7fb51290f700 1 Processor -- start 2024-03-21T12:47:01.391 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.464+0000 7fb51290f700 1 -- start start 2024-03-21T12:47:01.391 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.465+0000 7fb51290f700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb50c0792f0 con 0x7fb50c104e80 2024-03-21T12:47:01.391 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.465+0000 7fb50bfff700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb50c104e80 0x7fb50c19c7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57944/0 (socket says 172.21.15.43:57944) 2024-03-21T12:47:01.391 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.465+0000 7fb50bfff700 1 -- 172.21.15.43:0/2482735531 learned_addr learned my addr 172.21.15.43:0/2482735531 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:47:01.391 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.465+0000 7fb5097fa700 1 -- 172.21.15.43:0/2482735531 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1691919001 0 0) 0x7fb50c0792f0 con 0x7fb50c104e80 2024-03-21T12:47:01.391 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.465+0000 7fb5097fa700 1 -- 172.21.15.43:0/2482735531 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4f40055f0 con 0x7fb50c104e80 2024-03-21T12:47:01.391 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.465+0000 7fb5097fa700 1 -- 172.21.15.43:0/2482735531 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2821893241 0 0) 0x7fb4f40055f0 con 0x7fb50c104e80 2024-03-21T12:47:01.391 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.465+0000 7fb5097fa700 1 -- 172.21.15.43:0/2482735531 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb50c0792f0 con 0x7fb50c104e80 2024-03-21T12:47:01.391 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.465+0000 7fb5097fa700 1 -- 172.21.15.43:0/2482735531 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fb4fc001080 con 0x7fb50c104e80 2024-03-21T12:47:01.391 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.465+0000 7fb5097fa700 1 -- 172.21.15.43:0/2482735531 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 306807096 0 0) 0x7fb50c0792f0 con 0x7fb50c104e80 2024-03-21T12:47:01.391 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.465+0000 7fb5097fa700 1 -- 172.21.15.43:0/2482735531 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb50c0794c0 con 0x7fb50c104e80 2024-03-21T12:47:01.391 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.465+0000 7fb51290f700 1 -- 172.21.15.43:0/2482735531 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb50c0797b0 con 0x7fb50c104e80 2024-03-21T12:47:01.391 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.466+0000 7fb51290f700 1 -- 172.21.15.43:0/2482735531 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb50c19e190 con 0x7fb50c104e80 2024-03-21T12:47:01.391 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.466+0000 7fb5097fa700 1 -- 172.21.15.43:0/2482735531 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7fb4fc0059c0 con 0x7fb50c104e80 2024-03-21T12:47:01.391 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.466+0000 7fb5097fa700 1 -- 172.21.15.43:0/2482735531 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7fb4fc004930 con 0x7fb50c104e80 2024-03-21T12:47:01.392 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.466+0000 7fb5097fa700 1 -- 172.21.15.43:0/2482735531 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 13) v1 ==== 46315+0+0 (unknown 152183610 0 0) 0x7fb4fc0116e0 con 0x7fb50c104e80 2024-03-21T12:47:01.392 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.466+0000 7fb51290f700 1 -- 172.21.15.43:0/2482735531 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f8005150 con 0x7fb50c104e80 2024-03-21T12:47:01.392 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.467+0000 7fb5097fa700 1 -- 172.21.15.43:0/2482735531 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1006335234 0 0) 0x7fb4fc047c00 con 0x7fb50c104e80 2024-03-21T12:47:01.392 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.471+0000 7fb5097fa700 1 -- 172.21.15.43:0/2482735531 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7fb4fc017720 con 0x7fb50c104e80 2024-03-21T12:47:01.392 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.793+0000 7fb51290f700 1 -- 172.21.15.43:0/2482735531 --> v1:172.21.15.43:6789/0 -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fb4f8005440 con 0x7fb50c104e80 2024-03-21T12:47:01.392 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.795+0000 7fb5097fa700 1 -- 172.21.15.43:0/2482735531 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (unknown 1169358022 0 0) 0x7fb4fc017020 con 0x7fb50c104e80 2024-03-21T12:47:01.392 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.797+0000 7fb51290f700 1 -- 172.21.15.43:0/2482735531 >> v1:172.21.15.43:6800/4081968544 conn(0x7fb4f403a8f0 legacy=0x7fb4f403cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:01.392 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.797+0000 7fb51290f700 1 -- 172.21.15.43:0/2482735531 >> v1:172.21.15.43:6789/0 conn(0x7fb50c104e80 legacy=0x7fb50c19c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:01.392 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.797+0000 7fb51290f700 1 -- 172.21.15.43:0/2482735531 shutdown_connections 2024-03-21T12:47:01.392 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.797+0000 7fb51290f700 1 -- 172.21.15.43:0/2482735531 >> 172.21.15.43:0/2482735531 conn(0x7fb50c1004f0 msgr2=0x7fb50c18f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:47:01.392 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.797+0000 7fb51290f700 1 -- 172.21.15.43:0/2482735531 shutdown_connections 2024-03-21T12:47:01.392 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr 2024-03-21T12:47:00.797+0000 7fb51290f700 1 -- 172.21.15.43:0/2482735531 wait complete. 2024-03-21T12:47:01.392 INFO:teuthology.orchestra.run.smithi043.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-21T12:47:01.392 INFO:teuthology.orchestra.run.smithi043.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-21T12:47:01.392 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:47:01.393 INFO:teuthology.orchestra.run.smithi043.stdout: sudo /sbin/cephadm shell --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-21T12:47:01.393 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:47:01.393 INFO:teuthology.orchestra.run.smithi043.stdout:Or, if you are only running a single cluster on this host: 2024-03-21T12:47:01.393 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:47:01.393 INFO:teuthology.orchestra.run.smithi043.stdout: sudo /sbin/cephadm shell 2024-03-21T12:47:01.393 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:47:01.393 INFO:teuthology.orchestra.run.smithi043.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-21T12:47:01.393 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:47:01.393 INFO:teuthology.orchestra.run.smithi043.stdout: ceph telemetry on 2024-03-21T12:47:01.393 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:47:01.393 INFO:teuthology.orchestra.run.smithi043.stdout:For more information see: 2024-03-21T12:47:01.393 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:47:01.393 INFO:teuthology.orchestra.run.smithi043.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-21T12:47:01.393 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:47:01.393 INFO:teuthology.orchestra.run.smithi043.stdout:Bootstrap complete. 2024-03-21T12:47:01.431 INFO:tasks.cephadm:Fetching config... 2024-03-21T12:47:01.431 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-21T12:47:01.431 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-21T12:47:01.447 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-21T12:47:01.447 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-21T12:47:01.447 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-21T12:47:01.503 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-21T12:47:01.503 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-21T12:47:01.503 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-21T12:47:01.571 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-21T12:47:01.571 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-21T12:47:01.571 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-21T12:47:01.626 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-21T12:47:01.626 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFXsdcBQ76aLNx5bU9JoOV3qORr7R3smNyU0R24jPLQEMM26vALGGFzsB8oqNcVFuH4zlOu/DxuIRHmzGwfxq0BWIrCXpU+fUkkUD6n/GpK7z+zxRvl3r6vhqEMRaY4qM9UU0KhNpt5R9q/xarMk0b3D1q9j07ECjxzJxMtkNe0PA1Br920txh3qE64chMfPRgqx7X4/Za4vqrTbvysSNppLflsC1vpEEruPgTMEax1PHFuwuskgcq5ky/y4NpRLkI2Zxy9qxr8IJrQqHja0Cy/hQkM2frnb3eUli3DOrvon2jr6A7HiIv6TG3q96pHFSMDLEi1plIkrOTlN67sB5aoG7rVDeNb3g/fBVuE8srqafOGlN9/UDEDAYZnLtOPR+APa4oihpNN3XOlu0Apq05gwkhUKHyU2BcXLJx4zShPMbXuOgU1y606A7qd+Er+kMW6GGTCuZkCvYDABnkmmuhjL4MEPHW4t4bGefm/ltN+LWTQ11AnSiSPVMe6LIkQBs= ceph-c06c8d1a-e780-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-21T12:47:01.719 INFO:teuthology.orchestra.run.smithi043.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFXsdcBQ76aLNx5bU9JoOV3qORr7R3smNyU0R24jPLQEMM26vALGGFzsB8oqNcVFuH4zlOu/DxuIRHmzGwfxq0BWIrCXpU+fUkkUD6n/GpK7z+zxRvl3r6vhqEMRaY4qM9UU0KhNpt5R9q/xarMk0b3D1q9j07ECjxzJxMtkNe0PA1Br920txh3qE64chMfPRgqx7X4/Za4vqrTbvysSNppLflsC1vpEEruPgTMEax1PHFuwuskgcq5ky/y4NpRLkI2Zxy9qxr8IJrQqHja0Cy/hQkM2frnb3eUli3DOrvon2jr6A7HiIv6TG3q96pHFSMDLEi1plIkrOTlN67sB5aoG7rVDeNb3g/fBVuE8srqafOGlN9/UDEDAYZnLtOPR+APa4oihpNN3XOlu0Apq05gwkhUKHyU2BcXLJx4zShPMbXuOgU1y606A7qd+Er+kMW6GGTCuZkCvYDABnkmmuhjL4MEPHW4t4bGefm/ltN+LWTQ11AnSiSPVMe6LIkQBs= ceph-c06c8d1a-e780-11ee-95cd-87774f69a715 2024-03-21T12:47:01.734 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFXsdcBQ76aLNx5bU9JoOV3qORr7R3smNyU0R24jPLQEMM26vALGGFzsB8oqNcVFuH4zlOu/DxuIRHmzGwfxq0BWIrCXpU+fUkkUD6n/GpK7z+zxRvl3r6vhqEMRaY4qM9UU0KhNpt5R9q/xarMk0b3D1q9j07ECjxzJxMtkNe0PA1Br920txh3qE64chMfPRgqx7X4/Za4vqrTbvysSNppLflsC1vpEEruPgTMEax1PHFuwuskgcq5ky/y4NpRLkI2Zxy9qxr8IJrQqHja0Cy/hQkM2frnb3eUli3DOrvon2jr6A7HiIv6TG3q96pHFSMDLEi1plIkrOTlN67sB5aoG7rVDeNb3g/fBVuE8srqafOGlN9/UDEDAYZnLtOPR+APa4oihpNN3XOlu0Apq05gwkhUKHyU2BcXLJx4zShPMbXuOgU1y606A7qd+Er+kMW6GGTCuZkCvYDABnkmmuhjL4MEPHW4t4bGefm/ltN+LWTQ11AnSiSPVMe6LIkQBs= ceph-c06c8d1a-e780-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-21T12:47:01.780 INFO:teuthology.orchestra.run.smithi155.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFXsdcBQ76aLNx5bU9JoOV3qORr7R3smNyU0R24jPLQEMM26vALGGFzsB8oqNcVFuH4zlOu/DxuIRHmzGwfxq0BWIrCXpU+fUkkUD6n/GpK7z+zxRvl3r6vhqEMRaY4qM9UU0KhNpt5R9q/xarMk0b3D1q9j07ECjxzJxMtkNe0PA1Br920txh3qE64chMfPRgqx7X4/Za4vqrTbvysSNppLflsC1vpEEruPgTMEax1PHFuwuskgcq5ky/y4NpRLkI2Zxy9qxr8IJrQqHja0Cy/hQkM2frnb3eUli3DOrvon2jr6A7HiIv6TG3q96pHFSMDLEi1plIkrOTlN67sB5aoG7rVDeNb3g/fBVuE8srqafOGlN9/UDEDAYZnLtOPR+APa4oihpNN3XOlu0Apq05gwkhUKHyU2BcXLJx4zShPMbXuOgU1y606A7qd+Er+kMW6GGTCuZkCvYDABnkmmuhjL4MEPHW4t4bGefm/ltN+LWTQ11AnSiSPVMe6LIkQBs= ceph-c06c8d1a-e780-11ee-95cd-87774f69a715 2024-03-21T12:47:01.795 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-21T12:47:01.969 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:01 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2482735531' entity='client.admin' 2024-03-21T12:47:02.035 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:47:04.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.399+0000 7f44c0bf5700 1 -- 172.21.15.43:0/1480025530 >> v1:172.21.15.43:6789/0 conn(0x7f44bc100a20 legacy=0x7f44bc100e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:04.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.400+0000 7f44c0bf5700 1 -- 172.21.15.43:0/1480025530 shutdown_connections 2024-03-21T12:47:04.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.400+0000 7f44c0bf5700 1 -- 172.21.15.43:0/1480025530 >> 172.21.15.43:0/1480025530 conn(0x7f44bc0fbfc0 msgr2=0x7f44bc0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:47:04.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.400+0000 7f44c0bf5700 1 -- 172.21.15.43:0/1480025530 shutdown_connections 2024-03-21T12:47:04.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.400+0000 7f44c0bf5700 1 -- 172.21.15.43:0/1480025530 wait complete. 2024-03-21T12:47:04.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.400+0000 7f44c0bf5700 1 Processor -- start 2024-03-21T12:47:04.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.400+0000 7f44c0bf5700 1 -- start start 2024-03-21T12:47:04.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.401+0000 7f44c0bf5700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f44bc197050 con 0x7f44bc100a20 2024-03-21T12:47:04.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.401+0000 7f44ba59c700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f44bc100a20 0x7f44bc19b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55304/0 (socket says 172.21.15.43:55304) 2024-03-21T12:47:04.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.401+0000 7f44ba59c700 1 -- 172.21.15.43:0/2546290373 learned_addr learned my addr 172.21.15.43:0/2546290373 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:47:04.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.401+0000 7f44ab7fe700 1 -- 172.21.15.43:0/2546290373 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2912762600 0 0) 0x7f44bc197050 con 0x7f44bc100a20 2024-03-21T12:47:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.401+0000 7f44ab7fe700 1 -- 172.21.15.43:0/2546290373 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44a4005530 con 0x7f44bc100a20 2024-03-21T12:47:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.401+0000 7f44ab7fe700 1 -- 172.21.15.43:0/2546290373 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1153690713 0 0) 0x7f44a4005530 con 0x7f44bc100a20 2024-03-21T12:47:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.401+0000 7f44ab7fe700 1 -- 172.21.15.43:0/2546290373 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f44bc197050 con 0x7f44bc100a20 2024-03-21T12:47:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.401+0000 7f44ab7fe700 1 -- 172.21.15.43:0/2546290373 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f44ac001480 con 0x7f44bc100a20 2024-03-21T12:47:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.401+0000 7f44ab7fe700 1 -- 172.21.15.43:0/2546290373 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2435568244 0 0) 0x7f44bc197050 con 0x7f44bc100a20 2024-03-21T12:47:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.401+0000 7f44ab7fe700 1 -- 172.21.15.43:0/2546290373 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc06aa00 con 0x7f44bc100a20 2024-03-21T12:47:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.401+0000 7f44c0bf5700 1 -- 172.21.15.43:0/2546290373 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44bc06ad10 con 0x7f44bc100a20 2024-03-21T12:47:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.401+0000 7f44c0bf5700 1 -- 172.21.15.43:0/2546290373 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f44bc19cf50 con 0x7f44bc100a20 2024-03-21T12:47:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.401+0000 7f44ab7fe700 1 -- 172.21.15.43:0/2546290373 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f44ac001b60 con 0x7f44bc100a20 2024-03-21T12:47:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.402+0000 7f44ab7fe700 1 -- 172.21.15.43:0/2546290373 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f44ac005080 con 0x7f44bc100a20 2024-03-21T12:47:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.402+0000 7f44ab7fe700 1 -- 172.21.15.43:0/2546290373 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 13) v1 ==== 46315+0+0 (unknown 152183610 0 0) 0x7f44ac005300 con 0x7f44bc100a20 2024-03-21T12:47:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.402+0000 7f44c0bf5700 1 -- 172.21.15.43:0/2546290373 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f449c005150 con 0x7f44bc100a20 2024-03-21T12:47:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.403+0000 7f44ab7fe700 1 -- 172.21.15.43:0/2546290373 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1006335234 0 0) 0x7f44ac047c40 con 0x7f44bc100a20 2024-03-21T12:47:04.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.408+0000 7f44ab7fe700 1 -- 172.21.15.43:0/2546290373 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f44ac017680 con 0x7f44bc100a20 2024-03-21T12:47:04.594 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:04 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:04.595 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:04 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:04.595 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:04 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi043", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:47:04.595 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:04 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:04.595 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:04 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:47:04.595 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:04 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:04.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.615+0000 7f44c0bf5700 1 -- 172.21.15.43:0/2546290373 --> v1:172.21.15.43:6789/0 -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f449c005440 con 0x7f44bc100a20 2024-03-21T12:47:04.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.618+0000 7f44ab7fe700 1 -- 172.21.15.43:0/2546290373 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (unknown 3028693289 0 0) 0x7f44ac016f80 con 0x7f44bc100a20 2024-03-21T12:47:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.620+0000 7f44c0bf5700 1 -- 172.21.15.43:0/2546290373 >> v1:172.21.15.43:6800/4081968544 conn(0x7f44a403a500 legacy=0x7f44a403c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.620+0000 7f44c0bf5700 1 -- 172.21.15.43:0/2546290373 >> v1:172.21.15.43:6789/0 conn(0x7f44bc100a20 legacy=0x7f44bc19b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.620+0000 7f44c0bf5700 1 -- 172.21.15.43:0/2546290373 shutdown_connections 2024-03-21T12:47:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.620+0000 7f44c0bf5700 1 -- 172.21.15.43:0/2546290373 >> 172.21.15.43:0/2546290373 conn(0x7f44bc0fbfc0 msgr2=0x7f44bc0fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:47:04.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.621+0000 7f44c0bf5700 1 -- 172.21.15.43:0/2546290373 shutdown_connections 2024-03-21T12:47:04.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:04.621+0000 7f44c0bf5700 1 -- 172.21.15.43:0/2546290373 wait complete. 2024-03-21T12:47:05.250 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-21T12:47:05.251 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-21T12:47:05.759 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:05 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2546290373' entity='client.admin' 2024-03-21T12:47:05.759 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:05 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:47:05.759 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:05 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:05.773 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:47:07.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.254+0000 7f4327c8d700 1 -- 172.21.15.43:0/2548600778 >> v1:172.21.15.43:6789/0 conn(0x7f43201025b0 legacy=0x7f43201049a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:07.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.254+0000 7f4327c8d700 1 -- 172.21.15.43:0/2548600778 shutdown_connections 2024-03-21T12:47:07.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.254+0000 7f4327c8d700 1 -- 172.21.15.43:0/2548600778 >> 172.21.15.43:0/2548600778 conn(0x7f43200fbf80 msgr2=0x7f43200fe3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:47:07.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.254+0000 7f4327c8d700 1 -- 172.21.15.43:0/2548600778 shutdown_connections 2024-03-21T12:47:07.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.254+0000 7f4327c8d700 1 -- 172.21.15.43:0/2548600778 wait complete. 2024-03-21T12:47:07.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.255+0000 7f4327c8d700 1 Processor -- start 2024-03-21T12:47:07.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.255+0000 7f4327c8d700 1 -- start start 2024-03-21T12:47:07.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.255+0000 7f4327c8d700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4320196c30 con 0x7f43201025b0 2024-03-21T12:47:07.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.256+0000 7f4325a29700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f43201025b0 0x7f4320196520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55316/0 (socket says 172.21.15.43:55316) 2024-03-21T12:47:07.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.256+0000 7f4325a29700 1 -- 172.21.15.43:0/3689301951 learned_addr learned my addr 172.21.15.43:0/3689301951 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:47:07.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.256+0000 7f4316ffd700 1 -- 172.21.15.43:0/3689301951 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 200613491 0 0) 0x7f4320196c30 con 0x7f43201025b0 2024-03-21T12:47:07.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.256+0000 7f4316ffd700 1 -- 172.21.15.43:0/3689301951 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f430c005530 con 0x7f43201025b0 2024-03-21T12:47:07.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.256+0000 7f4316ffd700 1 -- 172.21.15.43:0/3689301951 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2898594125 0 0) 0x7f430c005530 con 0x7f43201025b0 2024-03-21T12:47:07.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.256+0000 7f4316ffd700 1 -- 172.21.15.43:0/3689301951 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4320196c30 con 0x7f43201025b0 2024-03-21T12:47:07.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.256+0000 7f4316ffd700 1 -- 172.21.15.43:0/3689301951 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f431c004750 con 0x7f43201025b0 2024-03-21T12:47:07.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.256+0000 7f4316ffd700 1 -- 172.21.15.43:0/3689301951 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4247358986 0 0) 0x7f4320196c30 con 0x7f43201025b0 2024-03-21T12:47:07.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.256+0000 7f4316ffd700 1 -- 172.21.15.43:0/3689301951 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4320196e00 con 0x7f43201025b0 2024-03-21T12:47:07.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.256+0000 7f4327c8d700 1 -- 172.21.15.43:0/3689301951 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43201970f0 con 0x7f43201025b0 2024-03-21T12:47:07.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.256+0000 7f4327c8d700 1 -- 172.21.15.43:0/3689301951 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f432019ac40 con 0x7f43201025b0 2024-03-21T12:47:07.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.256+0000 7f4316ffd700 1 -- 172.21.15.43:0/3689301951 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f431c003680 con 0x7f43201025b0 2024-03-21T12:47:07.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.256+0000 7f4316ffd700 1 -- 172.21.15.43:0/3689301951 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f431c004e40 con 0x7f43201025b0 2024-03-21T12:47:07.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.257+0000 7f4316ffd700 1 -- 172.21.15.43:0/3689301951 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 13) v1 ==== 46315+0+0 (unknown 152183610 0 0) 0x7f431c0050c0 con 0x7f43201025b0 2024-03-21T12:47:07.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.257+0000 7f4327c8d700 1 -- 172.21.15.43:0/3689301951 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4304005150 con 0x7f43201025b0 2024-03-21T12:47:07.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.258+0000 7f4316ffd700 1 -- 172.21.15.43:0/3689301951 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1006335234 0 0) 0x7f431c0488f0 con 0x7f43201025b0 2024-03-21T12:47:07.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.263+0000 7f4316ffd700 1 -- 172.21.15.43:0/3689301951 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f431c0175d0 con 0x7f43201025b0 2024-03-21T12:47:07.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.472+0000 7f4327c8d700 1 -- 172.21.15.43:0/3689301951 --> v1:172.21.15.43:6800/4081968544 -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f4304000bc0 con 0x7f430c03a550 2024-03-21T12:47:07.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.475+0000 7f4316ffd700 1 -- 172.21.15.43:0/3689301951 <== mgr.14152 v1:172.21.15.43:6800/4081968544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (unknown 0 0 0) 0x7f4304000bc0 con 0x7f430c03a550 2024-03-21T12:47:07.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.477+0000 7f4327c8d700 1 -- 172.21.15.43:0/3689301951 >> v1:172.21.15.43:6800/4081968544 conn(0x7f430c03a550 legacy=0x7f430c03ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:07.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.477+0000 7f4327c8d700 1 -- 172.21.15.43:0/3689301951 >> v1:172.21.15.43:6789/0 conn(0x7f43201025b0 legacy=0x7f4320196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:07.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.478+0000 7f4327c8d700 1 -- 172.21.15.43:0/3689301951 shutdown_connections 2024-03-21T12:47:07.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.478+0000 7f4327c8d700 1 -- 172.21.15.43:0/3689301951 >> 172.21.15.43:0/3689301951 conn(0x7f43200fbf80 msgr2=0x7f4320104ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:47:07.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.478+0000 7f4327c8d700 1 -- 172.21.15.43:0/3689301951 shutdown_connections 2024-03-21T12:47:07.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:07.478+0000 7f4327c8d700 1 -- 172.21.15.43:0/3689301951 wait complete. 2024-03-21T12:47:08.230 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi155 2024-03-21T12:47:08.230 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-21T12:47:08.230 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.conf 2024-03-21T12:47:08.248 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-21T12:47:08.249 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T12:47:08.306 INFO:tasks.cephadm:Adding host smithi155 to orchestrator... 2024-03-21T12:47:08.306 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph orch host add smithi155 2024-03-21T12:47:08.553 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:47:08.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:08 smithi043 ceph-mon[30062]: from='client.14176 v1:172.21.15.43:0/3689301951' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:47:08.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:08 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:08.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:08 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:47:08.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:08 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:08.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:08 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:47:08.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:08 smithi043 ceph-mon[30062]: Updating smithi043:/etc/ceph/ceph.conf 2024-03-21T12:47:08.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:08 smithi043 ceph-mon[30062]: Updating smithi043:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T12:47:08.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:08 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:09.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:09 smithi043 ceph-mon[30062]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-03-21T12:47:09.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:09 smithi043 ceph-mon[30062]: Updating smithi043:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T12:47:09.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:09 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:09.683 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:09 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:10.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.225+0000 7f6399564700 1 -- 172.21.15.43:0/4118456686 >> v1:172.21.15.43:6789/0 conn(0x7f63941069b0 legacy=0x7f6394106d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:10.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.225+0000 7f6399564700 1 -- 172.21.15.43:0/4118456686 shutdown_connections 2024-03-21T12:47:10.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.225+0000 7f6399564700 1 -- 172.21.15.43:0/4118456686 >> 172.21.15.43:0/4118456686 conn(0x7f63940fc4c0 msgr2=0x7f63940fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:47:10.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.225+0000 7f6399564700 1 -- 172.21.15.43:0/4118456686 shutdown_connections 2024-03-21T12:47:10.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.225+0000 7f6399564700 1 -- 172.21.15.43:0/4118456686 wait complete. 2024-03-21T12:47:10.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.226+0000 7f6399564700 1 Processor -- start 2024-03-21T12:47:10.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.226+0000 7f6399564700 1 -- start start 2024-03-21T12:47:10.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.226+0000 7f6399564700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6394199420 con 0x7f63941069b0 2024-03-21T12:47:10.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.226+0000 7f6392ffd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f63941069b0 0x7f6394198d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55322/0 (socket says 172.21.15.43:55322) 2024-03-21T12:47:10.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.226+0000 7f6392ffd700 1 -- 172.21.15.43:0/3945440632 learned_addr learned my addr 172.21.15.43:0/3945440632 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:47:10.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.227+0000 7f638bfff700 1 -- 172.21.15.43:0/3945440632 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1353784320 0 0) 0x7f6394199420 con 0x7f63941069b0 2024-03-21T12:47:10.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.227+0000 7f638bfff700 1 -- 172.21.15.43:0/3945440632 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6380005530 con 0x7f63941069b0 2024-03-21T12:47:10.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.227+0000 7f638bfff700 1 -- 172.21.15.43:0/3945440632 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4230658280 0 0) 0x7f6380005530 con 0x7f63941069b0 2024-03-21T12:47:10.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.227+0000 7f638bfff700 1 -- 172.21.15.43:0/3945440632 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6394199420 con 0x7f63941069b0 2024-03-21T12:47:10.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.227+0000 7f638bfff700 1 -- 172.21.15.43:0/3945440632 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f63840047e0 con 0x7f63941069b0 2024-03-21T12:47:10.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.227+0000 7f638bfff700 1 -- 172.21.15.43:0/3945440632 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 894715483 0 0) 0x7f6394199420 con 0x7f63941069b0 2024-03-21T12:47:10.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.227+0000 7f638bfff700 1 -- 172.21.15.43:0/3945440632 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63941995f0 con 0x7f63941069b0 2024-03-21T12:47:10.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.227+0000 7f6399564700 1 -- 172.21.15.43:0/3945440632 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63941998e0 con 0x7f63941069b0 2024-03-21T12:47:10.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.227+0000 7f6399564700 1 -- 172.21.15.43:0/3945440632 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f63941a1aa0 con 0x7f63941069b0 2024-03-21T12:47:10.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.227+0000 7f638bfff700 1 -- 172.21.15.43:0/3945440632 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f6384001a80 con 0x7f63941069b0 2024-03-21T12:47:10.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.227+0000 7f638bfff700 1 -- 172.21.15.43:0/3945440632 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f6384004d40 con 0x7f63941069b0 2024-03-21T12:47:10.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.228+0000 7f638bfff700 1 -- 172.21.15.43:0/3945440632 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 13) v1 ==== 46315+0+0 (unknown 152183610 0 0) 0x7f6384004f00 con 0x7f63941069b0 2024-03-21T12:47:10.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.228+0000 7f6399564700 1 -- 172.21.15.43:0/3945440632 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6374005120 con 0x7f63941069b0 2024-03-21T12:47:10.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.228+0000 7f638bfff700 1 -- 172.21.15.43:0/3945440632 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1006335234 0 0) 0x7f63840478f0 con 0x7f63941069b0 2024-03-21T12:47:10.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.233+0000 7f638bfff700 1 -- 172.21.15.43:0/3945440632 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f6394107530 con 0x7f63941069b0 2024-03-21T12:47:10.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:10.442+0000 7f6399564700 1 -- 172.21.15.43:0/3945440632 --> v1:172.21.15.43:6800/4081968544 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}) v1 -- 0x7f6374000b90 con 0x7f638003a690 2024-03-21T12:47:11.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:10 smithi043 ceph-mon[30062]: from='client.14178 v1:172.21.15.43:0/3945440632' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:47:12.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:11 smithi043 ceph-mon[30062]: Deploying cephadm binary to smithi155 2024-03-21T12:47:13.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:12 smithi043 ceph-mon[30062]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:15.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:15.019+0000 7f638bfff700 1 -- 172.21.15.43:0/3945440632 <== mgr.14152 v1:172.21.15.43:6800/4081968544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (unknown 0 0 2103545083) 0x7f6374000b90 con 0x7f638003a690 2024-03-21T12:47:15.021 INFO:teuthology.orchestra.run.smithi043.stdout:Added host 'smithi155' with addr '172.21.15.155' 2024-03-21T12:47:15.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:15.022+0000 7f6399564700 1 -- 172.21.15.43:0/3945440632 >> v1:172.21.15.43:6800/4081968544 conn(0x7f638003a690 legacy=0x7f638003cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:15.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:15.022+0000 7f6399564700 1 -- 172.21.15.43:0/3945440632 >> v1:172.21.15.43:6789/0 conn(0x7f63941069b0 legacy=0x7f6394198d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:15.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:15.022+0000 7f6399564700 1 -- 172.21.15.43:0/3945440632 shutdown_connections 2024-03-21T12:47:15.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:15.022+0000 7f6399564700 1 -- 172.21.15.43:0/3945440632 >> 172.21.15.43:0/3945440632 conn(0x7f63940fc4c0 msgr2=0x7f639406a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:47:15.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:15.022+0000 7f6399564700 1 -- 172.21.15.43:0/3945440632 shutdown_connections 2024-03-21T12:47:15.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:15.022+0000 7f6399564700 1 -- 172.21.15.43:0/3945440632 wait complete. 2024-03-21T12:47:15.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:14 smithi043 ceph-mon[30062]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:15.553 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-21T12:47:15.796 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:47:16.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:16 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:16.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:16 smithi043 ceph-mon[30062]: Added host smithi155 2024-03-21T12:47:16.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:16 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:47:16.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:16 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:17.159 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:17 smithi043 ceph-mon[30062]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:17.159 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:17 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:17.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.403+0000 7ff342121700 1 -- 172.21.15.43:0/3660192237 >> v1:172.21.15.43:6789/0 conn(0x7ff33c102c70 legacy=0x7ff33c103050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:17.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.403+0000 7ff342121700 1 -- 172.21.15.43:0/3660192237 shutdown_connections 2024-03-21T12:47:17.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.403+0000 7ff342121700 1 -- 172.21.15.43:0/3660192237 >> 172.21.15.43:0/3660192237 conn(0x7ff33c0fc500 msgr2=0x7ff33c0fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:47:17.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.403+0000 7ff342121700 1 -- 172.21.15.43:0/3660192237 shutdown_connections 2024-03-21T12:47:17.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.403+0000 7ff342121700 1 -- 172.21.15.43:0/3660192237 wait complete. 2024-03-21T12:47:17.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.404+0000 7ff342121700 1 Processor -- start 2024-03-21T12:47:17.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.404+0000 7ff342121700 1 -- start start 2024-03-21T12:47:17.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.404+0000 7ff342121700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff33c19cda0 con 0x7ff33c102c70 2024-03-21T12:47:17.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.404+0000 7ff33b7fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff33c102c70 0x7ff33c19c690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38008/0 (socket says 172.21.15.43:38008) 2024-03-21T12:47:17.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.404+0000 7ff33b7fe700 1 -- 172.21.15.43:0/3150102005 learned_addr learned my addr 172.21.15.43:0/3150102005 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:47:17.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.404+0000 7ff338ff9700 1 -- 172.21.15.43:0/3150102005 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3850084126 0 0) 0x7ff33c19cda0 con 0x7ff33c102c70 2024-03-21T12:47:17.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.404+0000 7ff338ff9700 1 -- 172.21.15.43:0/3150102005 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff328005530 con 0x7ff33c102c70 2024-03-21T12:47:17.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.405+0000 7ff338ff9700 1 -- 172.21.15.43:0/3150102005 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2191883453 0 0) 0x7ff328005530 con 0x7ff33c102c70 2024-03-21T12:47:17.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.405+0000 7ff338ff9700 1 -- 172.21.15.43:0/3150102005 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff33c19cda0 con 0x7ff33c102c70 2024-03-21T12:47:17.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.405+0000 7ff338ff9700 1 -- 172.21.15.43:0/3150102005 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7ff324004750 con 0x7ff33c102c70 2024-03-21T12:47:17.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.405+0000 7ff338ff9700 1 -- 172.21.15.43:0/3150102005 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 611481457 0 0) 0x7ff33c19cda0 con 0x7ff33c102c70 2024-03-21T12:47:17.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.405+0000 7ff338ff9700 1 -- 172.21.15.43:0/3150102005 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33c19cf70 con 0x7ff33c102c70 2024-03-21T12:47:17.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.405+0000 7ff342121700 1 -- 172.21.15.43:0/3150102005 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff33c19f420 con 0x7ff33c102c70 2024-03-21T12:47:17.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.405+0000 7ff342121700 1 -- 172.21.15.43:0/3150102005 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff33c19f7c0 con 0x7ff33c102c70 2024-03-21T12:47:17.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.405+0000 7ff338ff9700 1 -- 172.21.15.43:0/3150102005 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7ff324003700 con 0x7ff33c102c70 2024-03-21T12:47:17.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.405+0000 7ff338ff9700 1 -- 172.21.15.43:0/3150102005 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7ff324004c90 con 0x7ff33c102c70 2024-03-21T12:47:17.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.406+0000 7ff338ff9700 1 -- 172.21.15.43:0/3150102005 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 13) v1 ==== 46315+0+0 (unknown 152183610 0 0) 0x7ff324004f10 con 0x7ff33c102c70 2024-03-21T12:47:17.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.406+0000 7ff338ff9700 1 -- 172.21.15.43:0/3150102005 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1006335234 0 0) 0x7ff324048850 con 0x7ff33c102c70 2024-03-21T12:47:17.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.406+0000 7ff342121700 1 -- 172.21.15.43:0/3150102005 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff31c005120 con 0x7ff33c102c70 2024-03-21T12:47:17.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.412+0000 7ff338ff9700 1 -- 172.21.15.43:0/3150102005 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7ff324017470 con 0x7ff33c102c70 2024-03-21T12:47:17.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.621+0000 7ff342121700 1 -- 172.21.15.43:0/3150102005 --> v1:172.21.15.43:6800/4081968544 -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff31c000b90 con 0x7ff32803a5f0 2024-03-21T12:47:17.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.622+0000 7ff338ff9700 1 -- 172.21.15.43:0/3150102005 <== mgr.14152 v1:172.21.15.43:6800/4081968544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (unknown 0 0 3737234853) 0x7ff31c000b90 con 0x7ff32803a5f0 2024-03-21T12:47:17.622 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:47:17.623 INFO:teuthology.orchestra.run.smithi043.stdout:[{"addr": "172.21.15.43", "hostname": "smithi043", "labels": [], "status": ""}, {"addr": "172.21.15.155", "hostname": "smithi155", "labels": [], "status": ""}] 2024-03-21T12:47:17.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.624+0000 7ff342121700 1 -- 172.21.15.43:0/3150102005 >> v1:172.21.15.43:6800/4081968544 conn(0x7ff32803a5f0 legacy=0x7ff32803cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:17.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.624+0000 7ff342121700 1 -- 172.21.15.43:0/3150102005 >> v1:172.21.15.43:6789/0 conn(0x7ff33c102c70 legacy=0x7ff33c19c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:17.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.624+0000 7ff342121700 1 -- 172.21.15.43:0/3150102005 shutdown_connections 2024-03-21T12:47:17.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.624+0000 7ff342121700 1 -- 172.21.15.43:0/3150102005 >> 172.21.15.43:0/3150102005 conn(0x7ff33c0fc500 msgr2=0x7ff33c108220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:47:17.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.624+0000 7ff342121700 1 -- 172.21.15.43:0/3150102005 shutdown_connections 2024-03-21T12:47:17.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:17.624+0000 7ff342121700 1 -- 172.21.15.43:0/3150102005 wait complete. 2024-03-21T12:47:18.306 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-21T12:47:18.307 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph osd crush tunables default 2024-03-21T12:47:18.549 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:47:18.955 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:18 smithi043 ceph-mon[30062]: from='client.14180 v1:172.21.15.43:0/3150102005' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T12:47:18.955 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:18 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:19.864 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:19 smithi043 ceph-mon[30062]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.090+0000 7f8c49135700 1 -- 172.21.15.43:0/1791002289 >> v1:172.21.15.43:6789/0 conn(0x7f8c44107ac0 legacy=0x7f8c44107ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.090+0000 7f8c49135700 1 -- 172.21.15.43:0/1791002289 shutdown_connections 2024-03-21T12:47:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.090+0000 7f8c49135700 1 -- 172.21.15.43:0/1791002289 >> 172.21.15.43:0/1791002289 conn(0x7f8c440fd5d0 msgr2=0x7f8c440ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:47:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.090+0000 7f8c49135700 1 -- 172.21.15.43:0/1791002289 shutdown_connections 2024-03-21T12:47:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.090+0000 7f8c49135700 1 -- 172.21.15.43:0/1791002289 wait complete. 2024-03-21T12:47:20.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.091+0000 7f8c49135700 1 Processor -- start 2024-03-21T12:47:20.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.091+0000 7f8c49135700 1 -- start start 2024-03-21T12:47:20.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.091+0000 7f8c49135700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8c4419eff0 con 0x7f8c44107ac0 2024-03-21T12:47:20.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.091+0000 7f8c42d9d700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8c44107ac0 0x7f8c4419e8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38032/0 (socket says 172.21.15.43:38032) 2024-03-21T12:47:20.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.091+0000 7f8c42d9d700 1 -- 172.21.15.43:0/2835169065 learned_addr learned my addr 172.21.15.43:0/2835169065 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:47:20.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.091+0000 7f8c3bfff700 1 -- 172.21.15.43:0/2835169065 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 63275663 0 0) 0x7f8c4419eff0 con 0x7f8c44107ac0 2024-03-21T12:47:20.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.091+0000 7f8c3bfff700 1 -- 172.21.15.43:0/2835169065 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c30005530 con 0x7f8c44107ac0 2024-03-21T12:47:20.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.092+0000 7f8c3bfff700 1 -- 172.21.15.43:0/2835169065 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2185808877 0 0) 0x7f8c30005530 con 0x7f8c44107ac0 2024-03-21T12:47:20.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.092+0000 7f8c3bfff700 1 -- 172.21.15.43:0/2835169065 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8c4419eff0 con 0x7f8c44107ac0 2024-03-21T12:47:20.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.092+0000 7f8c3bfff700 1 -- 172.21.15.43:0/2835169065 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f8c34004b60 con 0x7f8c44107ac0 2024-03-21T12:47:20.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.092+0000 7f8c3bfff700 1 -- 172.21.15.43:0/2835169065 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3450467978 0 0) 0x7f8c4419eff0 con 0x7f8c44107ac0 2024-03-21T12:47:20.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.092+0000 7f8c3bfff700 1 -- 172.21.15.43:0/2835169065 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4419f1c0 con 0x7f8c44107ac0 2024-03-21T12:47:20.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.092+0000 7f8c49135700 1 -- 172.21.15.43:0/2835169065 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c4419f4b0 con 0x7f8c44107ac0 2024-03-21T12:47:20.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.092+0000 7f8c49135700 1 -- 172.21.15.43:0/2835169065 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c441a1a20 con 0x7f8c44107ac0 2024-03-21T12:47:20.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.092+0000 7f8c3bfff700 1 -- 172.21.15.43:0/2835169065 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f8c34001e00 con 0x7f8c44107ac0 2024-03-21T12:47:20.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.092+0000 7f8c3bfff700 1 -- 172.21.15.43:0/2835169065 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f8c340050b0 con 0x7f8c44107ac0 2024-03-21T12:47:20.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.093+0000 7f8c3bfff700 1 -- 172.21.15.43:0/2835169065 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 13) v1 ==== 46315+0+0 (unknown 152183610 0 0) 0x7f8c34005270 con 0x7f8c44107ac0 2024-03-21T12:47:20.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.093+0000 7f8c49135700 1 -- 172.21.15.43:0/2835169065 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c24005120 con 0x7f8c44107ac0 2024-03-21T12:47:20.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.093+0000 7f8c3bfff700 1 -- 172.21.15.43:0/2835169065 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (unknown 1006335234 0 0) 0x7f8c34001760 con 0x7f8c44107ac0 2024-03-21T12:47:20.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.098+0000 7f8c3bfff700 1 -- 172.21.15.43:0/2835169065 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f8c44108640 con 0x7f8c44107ac0 2024-03-21T12:47:20.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.307+0000 7f8c49135700 1 -- 172.21.15.43:0/2835169065 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f8c24005410 con 0x7f8c44107ac0 2024-03-21T12:47:20.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.642+0000 7f8c3bfff700 1 -- 172.21.15.43:0/2835169065 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (unknown 3126668360 0 0) 0x7f8c34017730 con 0x7f8c44107ac0 2024-03-21T12:47:20.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.645+0000 7f8c49135700 1 -- 172.21.15.43:0/2835169065 >> v1:172.21.15.43:6800/4081968544 conn(0x7f8c3003a640 legacy=0x7f8c3003cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:20.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.645+0000 7f8c49135700 1 -- 172.21.15.43:0/2835169065 >> v1:172.21.15.43:6789/0 conn(0x7f8c44107ac0 legacy=0x7f8c4419e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:20.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.645+0000 7f8c49135700 1 -- 172.21.15.43:0/2835169065 shutdown_connections 2024-03-21T12:47:20.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.645+0000 7f8c49135700 1 -- 172.21.15.43:0/2835169065 >> 172.21.15.43:0/2835169065 conn(0x7f8c440fd5d0 msgr2=0x7f8c44109f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:47:20.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.645+0000 7f8c49135700 1 -- 172.21.15.43:0/2835169065 shutdown_connections 2024-03-21T12:47:20.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:20.645+0000 7f8c49135700 1 -- 172.21.15.43:0/2835169065 wait complete. 2024-03-21T12:47:20.646 INFO:teuthology.orchestra.run.smithi043.stderr:adjusted tunables profile to default 2024-03-21T12:47:20.964 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:20 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2835169065' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-21T12:47:21.299 INFO:tasks.cephadm:Adding mon.a on smithi043 2024-03-21T12:47:21.300 INFO:tasks.cephadm:Adding mon.c on smithi043 2024-03-21T12:47:21.300 INFO:tasks.cephadm:Adding mon.b on smithi155 2024-03-21T12:47:21.300 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph orch apply mon '3;smithi043:[v1:172.21.15.43:6789]=a;smithi043:[v1:172.21.15.43:6790]=c;smithi155:[v1:172.21.15.155:6789]=b' 2024-03-21T12:47:21.534 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T12:47:21.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:21 smithi043 ceph-mon[30062]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:21 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2835169065' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-21T12:47:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:21 smithi043 ceph-mon[30062]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T12:47:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:21 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:21 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:21 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:21 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:21 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:47:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:21 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:21 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:47:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:21 smithi043 ceph-mon[30062]: Updating smithi155:/etc/ceph/ceph.conf 2024-03-21T12:47:22.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:22 smithi043 ceph-mon[30062]: Updating smithi155:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T12:47:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:22 smithi043 ceph-mon[30062]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-03-21T12:47:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:22 smithi043 ceph-mon[30062]: Updating smithi155:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T12:47:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:22 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:22 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:22 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:23.161 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.160+0000 7f5b86c16700 1 -- 172.21.15.155:0/2775986813 >> v1:172.21.15.43:6789/0 conn(0x7f5b80106a90 legacy=0x7f5b80106e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:23.161 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.160+0000 7f5b86c16700 1 -- 172.21.15.155:0/2775986813 shutdown_connections 2024-03-21T12:47:23.161 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.160+0000 7f5b86c16700 1 -- 172.21.15.155:0/2775986813 >> 172.21.15.155:0/2775986813 conn(0x7f5b800fbc20 msgr2=0x7f5b800fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:47:23.161 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.161+0000 7f5b86c16700 1 -- 172.21.15.155:0/2775986813 shutdown_connections 2024-03-21T12:47:23.161 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.161+0000 7f5b86c16700 1 -- 172.21.15.155:0/2775986813 wait complete. 2024-03-21T12:47:23.161 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.161+0000 7f5b86c16700 1 Processor -- start 2024-03-21T12:47:23.162 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.161+0000 7f5b86c16700 1 -- start start 2024-03-21T12:47:23.162 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.161+0000 7f5b86c16700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5b8019ee70 con 0x7f5b80106a90 2024-03-21T12:47:23.163 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.162+0000 7f5b849b2700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5b80106a90 0x7f5b8019e760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.155:58590/0 (socket says 172.21.15.155:58590) 2024-03-21T12:47:23.163 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.162+0000 7f5b849b2700 1 -- 172.21.15.155:0/3986417470 learned_addr learned my addr 172.21.15.155:0/3986417470 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-03-21T12:47:23.163 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.162+0000 7f5b7dffb700 1 -- 172.21.15.155:0/3986417470 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 461616507 0 0) 0x7f5b8019ee70 con 0x7f5b80106a90 2024-03-21T12:47:23.164 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.162+0000 7f5b7dffb700 1 -- 172.21.15.155:0/3986417470 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b70005580 con 0x7f5b80106a90 2024-03-21T12:47:23.164 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.162+0000 7f5b7dffb700 1 -- 172.21.15.155:0/3986417470 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2175315119 0 0) 0x7f5b70005580 con 0x7f5b80106a90 2024-03-21T12:47:23.164 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.162+0000 7f5b7dffb700 1 -- 172.21.15.155:0/3986417470 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5b8019ee70 con 0x7f5b80106a90 2024-03-21T12:47:23.164 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.162+0000 7f5b7dffb700 1 -- 172.21.15.155:0/3986417470 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f5b740018e0 con 0x7f5b80106a90 2024-03-21T12:47:23.164 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.162+0000 7f5b7dffb700 1 -- 172.21.15.155:0/3986417470 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2878938804 0 0) 0x7f5b8019ee70 con 0x7f5b80106a90 2024-03-21T12:47:23.164 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.162+0000 7f5b7dffb700 1 -- 172.21.15.155:0/3986417470 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8019f040 con 0x7f5b80106a90 2024-03-21T12:47:23.164 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.162+0000 7f5b86c16700 1 -- 172.21.15.155:0/3986417470 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b8019f330 con 0x7f5b80106a90 2024-03-21T12:47:23.164 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.162+0000 7f5b7dffb700 1 -- 172.21.15.155:0/3986417470 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f5b74004700 con 0x7f5b80106a90 2024-03-21T12:47:23.164 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.162+0000 7f5b7dffb700 1 -- 172.21.15.155:0/3986417470 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f5b74004e30 con 0x7f5b80106a90 2024-03-21T12:47:23.164 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.163+0000 7f5b86c16700 1 -- 172.21.15.155:0/3986417470 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b801a18a0 con 0x7f5b80106a90 2024-03-21T12:47:23.164 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.163+0000 7f5b7dffb700 1 -- 172.21.15.155:0/3986417470 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 13) v1 ==== 46315+0+0 (unknown 152183610 0 0) 0x7f5b74005120 con 0x7f5b80106a90 2024-03-21T12:47:23.164 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.163+0000 7f5b86c16700 1 -- 172.21.15.155:0/3986417470 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b800637c0 con 0x7f5b80106a90 2024-03-21T12:47:23.164 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.164+0000 7f5b7dffb700 1 -- 172.21.15.155:0/3986417470 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (unknown 709340102 0 0) 0x7f5b74047cc0 con 0x7f5b80106a90 2024-03-21T12:47:23.168 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.169+0000 7f5b7dffb700 1 -- 172.21.15.155:0/3986417470 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f5b74017420 con 0x7f5b80106a90 2024-03-21T12:47:23.372 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.372+0000 7f5b86c16700 1 -- 172.21.15.155:0/3986417470 --> v1:172.21.15.43:6800/4081968544 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:[v1:172.21.15.43:6789]=a;smithi043:[v1:172.21.15.43:6790]=c;smithi155:[v1:172.21.15.155:6789]=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b8010c840 con 0x7f5b7003a570 2024-03-21T12:47:23.378 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.379+0000 7f5b7dffb700 1 -- 172.21.15.155:0/3986417470 <== mgr.14152 v1:172.21.15.43:6800/4081968544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 3265049985) 0x7f5b8010c840 con 0x7f5b7003a570 2024-03-21T12:47:23.378 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled mon update... 2024-03-21T12:47:23.380 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.381+0000 7f5b86c16700 1 -- 172.21.15.155:0/3986417470 >> v1:172.21.15.43:6800/4081968544 conn(0x7f5b7003a570 legacy=0x7f5b7003ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:23.381 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.381+0000 7f5b86c16700 1 -- 172.21.15.155:0/3986417470 >> v1:172.21.15.43:6789/0 conn(0x7f5b80106a90 legacy=0x7f5b8019e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:23.381 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.381+0000 7f5b86c16700 1 -- 172.21.15.155:0/3986417470 shutdown_connections 2024-03-21T12:47:23.381 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.381+0000 7f5b86c16700 1 -- 172.21.15.155:0/3986417470 >> 172.21.15.155:0/3986417470 conn(0x7f5b800fbc20 msgr2=0x7f5b80108d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:47:23.381 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.381+0000 7f5b86c16700 1 -- 172.21.15.155:0/3986417470 shutdown_connections 2024-03-21T12:47:23.381 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:23.381+0000 7f5b86c16700 1 -- 172.21.15.155:0/3986417470 wait complete. 2024-03-21T12:47:23.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:23 smithi043 ceph-mon[30062]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:23.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:23 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:23.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:23 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:47:23.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:23 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:23.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:23 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:47:23.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:23 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:23.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:23 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T12:47:23.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:23 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:23.991 DEBUG:teuthology.orchestra.run.smithi043:mon.c> sudo journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mon.c.service 2024-03-21T12:47:23.994 DEBUG:teuthology.orchestra.run.smithi155:mon.b> sudo journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mon.b.service 2024-03-21T12:47:23.996 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-21T12:47:23.996 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T12:47:24.745 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T12:47:24.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:24 smithi043 ceph-mon[30062]: from='client.14184 v1:172.21.15.155:0/3986417470' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:[v1:172.21.15.43:6789]=a;smithi043:[v1:172.21.15.43:6790]=c;smithi155:[v1:172.21.15.155:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:47:24.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:24 smithi043 ceph-mon[30062]: Saving service mon spec with placement smithi043:[v1:172.21.15.43:6789]=a;smithi043:[v1:172.21.15.43:6790]=c;smithi155:[v1:172.21.15.155:6789]=b;count:3 2024-03-21T12:47:24.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:24 smithi043 ceph-mon[30062]: Deploying daemon mon.b on smithi155 2024-03-21T12:47:25.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:25 smithi043 ceph-mon[30062]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:27.110 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.109+0000 7f032acd5700 1 -- 172.21.15.155:0/2542437224 >> v1:172.21.15.43:6789/0 conn(0x7f0324107a80 legacy=0x7f0324107e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:27.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.109+0000 7f032acd5700 1 -- 172.21.15.155:0/2542437224 shutdown_connections 2024-03-21T12:47:27.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.109+0000 7f032acd5700 1 -- 172.21.15.155:0/2542437224 >> 172.21.15.155:0/2542437224 conn(0x7f03240fd590 msgr2=0x7f03240ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:47:27.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.109+0000 7f032acd5700 1 -- 172.21.15.155:0/2542437224 shutdown_connections 2024-03-21T12:47:27.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.109+0000 7f032acd5700 1 -- 172.21.15.155:0/2542437224 wait complete. 2024-03-21T12:47:27.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.109+0000 7f032acd5700 1 Processor -- start 2024-03-21T12:47:27.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.109+0000 7f032acd5700 1 -- start start 2024-03-21T12:47:27.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.109+0000 7f032acd5700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f032419cde0 con 0x7f0324107a80 2024-03-21T12:47:27.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.110+0000 7f0328a71700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0324107a80 0x7f032419c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.155:58610/0 (socket says 172.21.15.155:58610) 2024-03-21T12:47:27.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.110+0000 7f0328a71700 1 -- 172.21.15.155:0/2625709698 learned_addr learned my addr 172.21.15.155:0/2625709698 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-03-21T12:47:27.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.110+0000 7f0321ffb700 1 -- 172.21.15.155:0/2625709698 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2639494360 0 0) 0x7f032419cde0 con 0x7f0324107a80 2024-03-21T12:47:27.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.110+0000 7f0321ffb700 1 -- 172.21.15.155:0/2625709698 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0314005530 con 0x7f0324107a80 2024-03-21T12:47:27.111 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.110+0000 7f0321ffb700 1 -- 172.21.15.155:0/2625709698 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1025017388 0 0) 0x7f0314005530 con 0x7f0324107a80 2024-03-21T12:47:27.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.110+0000 7f0321ffb700 1 -- 172.21.15.155:0/2625709698 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f032419cde0 con 0x7f0324107a80 2024-03-21T12:47:27.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.110+0000 7f0321ffb700 1 -- 172.21.15.155:0/2625709698 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f03180047e0 con 0x7f0324107a80 2024-03-21T12:47:27.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.110+0000 7f0321ffb700 1 -- 172.21.15.155:0/2625709698 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2170419107 0 0) 0x7f032419cde0 con 0x7f0324107a80 2024-03-21T12:47:27.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.110+0000 7f0321ffb700 1 -- 172.21.15.155:0/2625709698 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032419cfb0 con 0x7f0324107a80 2024-03-21T12:47:27.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.110+0000 7f032acd5700 1 -- 172.21.15.155:0/2625709698 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f032419d2a0 con 0x7f0324107a80 2024-03-21T12:47:27.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.111+0000 7f0321ffb700 1 -- 172.21.15.155:0/2625709698 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f0318003b10 con 0x7f0324107a80 2024-03-21T12:47:27.112 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.111+0000 7f0321ffb700 1 -- 172.21.15.155:0/2625709698 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1286547744 0 0) 0x7f0318005030 con 0x7f0324107a80 2024-03-21T12:47:27.113 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.111+0000 7f032acd5700 1 -- 172.21.15.155:0/2625709698 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f032419f810 con 0x7f0324107a80 2024-03-21T12:47:27.113 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.111+0000 7f0321ffb700 1 -- 172.21.15.155:0/2625709698 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 13) v1 ==== 46315+0+0 (unknown 152183610 0 0) 0x7f0318005300 con 0x7f0324107a80 2024-03-21T12:47:27.113 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.111+0000 7f032acd5700 1 -- 172.21.15.155:0/2625709698 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03240637c0 con 0x7f0324107a80 2024-03-21T12:47:27.113 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.112+0000 7f0321ffb700 1 -- 172.21.15.155:0/2625709698 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (unknown 709340102 0 0) 0x7f0318047cc0 con 0x7f0324107a80 2024-03-21T12:47:27.116 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.117+0000 7f0321ffb700 1 -- 172.21.15.155:0/2625709698 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f0324108600 con 0x7f0324107a80 2024-03-21T12:47:27.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.398+0000 7f032acd5700 1 -- 172.21.15.155:0/2625709698 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f032410c740 con 0x7f0324107a80 2024-03-21T12:47:27.398 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.399+0000 7f0321ffb700 1 -- 172.21.15.155:0/2625709698 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+694 (unknown 2237029548 0 2302938956) 0x7f031801efd0 con 0x7f0324107a80 2024-03-21T12:47:27.398 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:47:27.399 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"c06c8d1a-e780-11ee-95cd-87774f69a715","modified":"2024-03-21T12:45:43.688830Z","created":"2024-03-21T12:45:43.688830Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-21T12:47:27.400 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.401+0000 7f032acd5700 1 -- 172.21.15.155:0/2625709698 >> v1:172.21.15.43:6800/4081968544 conn(0x7f031403a550 legacy=0x7f031403ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:27.401 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.401+0000 7f032acd5700 1 -- 172.21.15.155:0/2625709698 >> v1:172.21.15.43:6789/0 conn(0x7f0324107a80 legacy=0x7f032419c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:27.401 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.401+0000 7f032acd5700 1 -- 172.21.15.155:0/2625709698 shutdown_connections 2024-03-21T12:47:27.401 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.401+0000 7f032acd5700 1 -- 172.21.15.155:0/2625709698 >> 172.21.15.155:0/2625709698 conn(0x7f03240fd590 msgr2=0x7f0324109a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:47:27.401 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.401+0000 7f032acd5700 1 -- 172.21.15.155:0/2625709698 shutdown_connections 2024-03-21T12:47:27.401 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:27.401+0000 7f032acd5700 1 -- 172.21.15.155:0/2625709698 wait complete. 2024-03-21T12:47:27.401 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-03-21T12:47:27.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:27 smithi043 ceph-mon[30062]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:27.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:27 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.155:0/2625709698' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T12:47:29.072 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-21T12:47:29.072 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T12:47:29.097 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:28 smithi155 systemd[1]: Started Ceph mon.b for c06c8d1a-e780-11ee-95cd-87774f69a715. 2024-03-21T12:47:29.307 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T12:47:33.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.979+0000 7fbc23ee2700 1 -- 172.21.15.155:0/2018694764 >> v1:172.21.15.43:6789/0 conn(0x7fbc000035f0 legacy=0x7fbc00005a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:33.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.979+0000 7fbc23ee2700 1 -- 172.21.15.155:0/2018694764 shutdown_connections 2024-03-21T12:47:33.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.979+0000 7fbc23ee2700 1 -- 172.21.15.155:0/2018694764 >> 172.21.15.155:0/2018694764 conn(0x7fbc1c0fd740 msgr2=0x7fbc1c0ffb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:47:33.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.980+0000 7fbc23ee2700 1 -- 172.21.15.155:0/2018694764 shutdown_connections 2024-03-21T12:47:33.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.980+0000 7fbc23ee2700 1 -- 172.21.15.155:0/2018694764 wait complete. 2024-03-21T12:47:33.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.981+0000 7fbc23ee2700 1 Processor -- start 2024-03-21T12:47:33.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.981+0000 7fbc23ee2700 1 -- start start 2024-03-21T12:47:33.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.981+0000 7fbc23ee2700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbc1c075030 con 0x7fbc000035f0 2024-03-21T12:47:33.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.981+0000 7fbc23ee2700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbc1c075200 con 0x7fbc1c074ba0 2024-03-21T12:47:33.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.981+0000 7fbc2147d700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fbc1c074ba0 0x7fbc1c1a3ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:46446/0 (socket says 172.21.15.155:46446) 2024-03-21T12:47:33.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.981+0000 7fbc2147d700 1 -- 172.21.15.155:0/2264383897 learned_addr learned my addr 172.21.15.155:0/2264383897 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-03-21T12:47:33.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.982+0000 7fbc12ffd700 1 -- 172.21.15.155:0/2264383897 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1886849218 0 0) 0x7fbc1c075030 con 0x7fbc000035f0 2024-03-21T12:47:33.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.982+0000 7fbc12ffd700 1 -- 172.21.15.155:0/2264383897 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc080035b0 con 0x7fbc000035f0 2024-03-21T12:47:33.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.982+0000 7fbc12ffd700 1 -- 172.21.15.155:0/2264383897 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 150716857 0 0) 0x7fbc080035b0 con 0x7fbc000035f0 2024-03-21T12:47:33.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.982+0000 7fbc12ffd700 1 -- 172.21.15.155:0/2264383897 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbc1c075030 con 0x7fbc000035f0 2024-03-21T12:47:33.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.982+0000 7fbc12ffd700 1 -- 172.21.15.155:0/2264383897 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 239+0+0 (unknown 4138250928 0 0) 0x7fbc180046f0 con 0x7fbc000035f0 2024-03-21T12:47:33.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.982+0000 7fbc12ffd700 1 -- 172.21.15.155:0/2264383897 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 209910403 0 0) 0x7fbc1c075030 con 0x7fbc000035f0 2024-03-21T12:47:33.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.982+0000 7fbc12ffd700 1 -- 172.21.15.155:0/2264383897 >> v1:172.21.15.155:6789/0 conn(0x7fbc1c074ba0 legacy=0x7fbc1c1a3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:33.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.983+0000 7fbc12ffd700 1 -- 172.21.15.155:0/2264383897 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc1c1a71f0 con 0x7fbc000035f0 2024-03-21T12:47:33.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.983+0000 7fbc23ee2700 1 -- 172.21.15.155:0/2264383897 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1c1a51d0 con 0x7fbc000035f0 2024-03-21T12:47:33.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.983+0000 7fbc23ee2700 1 -- 172.21.15.155:0/2264383897 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1c1a56d0 con 0x7fbc000035f0 2024-03-21T12:47:33.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.984+0000 7fbc23ee2700 1 -- 172.21.15.155:0/2264383897 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc1c103330 con 0x7fbc000035f0 2024-03-21T12:47:33.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.984+0000 7fbc12ffd700 1 -- 172.21.15.155:0/2264383897 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7fbc18001b50 con 0x7fbc000035f0 2024-03-21T12:47:33.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.984+0000 7fbc12ffd700 1 -- 172.21.15.155:0/2264383897 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 239+0+0 (unknown 4138250928 0 0) 0x7fbc18004c00 con 0x7fbc000035f0 2024-03-21T12:47:33.985 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.985+0000 7fbc12ffd700 1 -- 172.21.15.155:0/2264383897 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 13) v1 ==== 46315+0+0 (unknown 152183610 0 0) 0x7fbc180102a0 con 0x7fbc000035f0 2024-03-21T12:47:33.985 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.985+0000 7fbc12ffd700 1 -- 172.21.15.155:0/2264383897 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (unknown 709340102 0 0) 0x7fbc18047560 con 0x7fbc000035f0 2024-03-21T12:47:33.989 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:33.990+0000 7fbc12ffd700 1 -- 172.21.15.155:0/2264383897 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7fbc18016180 con 0x7fbc000035f0 2024-03-21T12:47:34.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:34.267+0000 7fbc23ee2700 1 -- 172.21.15.155:0/2264383897 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbc1c0637c0 con 0x7fbc000035f0 2024-03-21T12:47:34.267 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:34 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T12:47:34.267 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:34 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:47:34.267 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:34 smithi155 ceph-mon[33846]: mon.a calling monitor election 2024-03-21T12:47:34.267 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:34 smithi155 ceph-mon[33846]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:34.267 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:34 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:47:34.267 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:34 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:47:34.267 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:34 smithi155 ceph-mon[33846]: mon.b calling monitor election 2024-03-21T12:47:34.267 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:34 smithi155 ceph-mon[33846]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:34.267 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:34 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:47:34.267 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:34 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:47:34.268 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:34 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:47:34.268 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:34 smithi155 ceph-mon[33846]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-03-21T12:47:34.268 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:34 smithi155 ceph-mon[33846]: monmap e2: 2 mons at {a=v1:172.21.15.43:6789/0,b=v1:172.21.15.155:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-03-21T12:47:34.268 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:34 smithi155 ceph-mon[33846]: fsmap 2024-03-21T12:47:34.268 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:34 smithi155 ceph-mon[33846]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T12:47:34.268 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:34 smithi155 ceph-mon[33846]: mgrmap e13: y(active, since 42s) 2024-03-21T12:47:34.268 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:34 smithi155 ceph-mon[33846]: overall HEALTH_OK 2024-03-21T12:47:34.268 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:34 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:34.268 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:34 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:34.268 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:34 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:34.268 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:34 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T12:47:34.268 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:34 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:34.269 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:34.267+0000 7fbc12ffd700 1 -- 172.21.15.155:0/2264383897 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+912 (unknown 3557084514 0 1071210776) 0x7fbc18015a80 con 0x7fbc000035f0 2024-03-21T12:47:34.269 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:47:34.269 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":2,"fsid":"c06c8d1a-e780-11ee-95cd-87774f69a715","modified":"2024-03-21T12:47:28.940301Z","created":"2024-03-21T12:45:43.688830Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6789","nonce":0}]},"addr":"172.21.15.155:6789/0","public_addr":"172.21.15.155:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-21T12:47:34.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:34.269+0000 7fbc23ee2700 1 -- 172.21.15.155:0/2264383897 >> v1:172.21.15.43:6800/4081968544 conn(0x7fbc0803a590 legacy=0x7fbc0803ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:34.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:34.269+0000 7fbc23ee2700 1 -- 172.21.15.155:0/2264383897 >> v1:172.21.15.43:6789/0 conn(0x7fbc000035f0 legacy=0x7fbc1c074490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:34.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:34.269+0000 7fbc23ee2700 1 -- 172.21.15.155:0/2264383897 shutdown_connections 2024-03-21T12:47:34.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:34.269+0000 7fbc23ee2700 1 -- 172.21.15.155:0/2264383897 >> 172.21.15.155:0/2264383897 conn(0x7fbc1c0fd740 msgr2=0x7fbc1c0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:47:34.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:34.269+0000 7fbc23ee2700 1 -- 172.21.15.155:0/2264383897 shutdown_connections 2024-03-21T12:47:34.270 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:34.269+0000 7fbc23ee2700 1 -- 172.21.15.155:0/2264383897 wait complete. 2024-03-21T12:47:34.270 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 2 2024-03-21T12:47:34.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:34 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T12:47:34.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:34 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:47:34.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:34 smithi043 ceph-mon[30062]: mon.a calling monitor election 2024-03-21T12:47:34.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:34 smithi043 ceph-mon[30062]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:34.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:34 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:47:34.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:34 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:47:34.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:34 smithi043 ceph-mon[30062]: mon.b calling monitor election 2024-03-21T12:47:34.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:34 smithi043 ceph-mon[30062]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:34.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:34 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:47:34.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:34 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:47:34.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:34 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:47:34.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:34 smithi043 ceph-mon[30062]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-03-21T12:47:34.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:34 smithi043 ceph-mon[30062]: monmap e2: 2 mons at {a=v1:172.21.15.43:6789/0,b=v1:172.21.15.155:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-03-21T12:47:34.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:34 smithi043 ceph-mon[30062]: fsmap 2024-03-21T12:47:34.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:34 smithi043 ceph-mon[30062]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T12:47:34.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:34 smithi043 ceph-mon[30062]: mgrmap e13: y(active, since 42s) 2024-03-21T12:47:34.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:34 smithi043 ceph-mon[30062]: overall HEALTH_OK 2024-03-21T12:47:34.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:34 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:34.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:34 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:34.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:34 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:34.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:34 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T12:47:34.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:34 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:35.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:35 smithi155 ceph-mon[33846]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:35.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:35 smithi155 ceph-mon[33846]: Deploying daemon mon.c on smithi043 2024-03-21T12:47:35.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:35 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.155:0/2264383897' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T12:47:35.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:35 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:47:35.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:35 smithi043 ceph-mon[30062]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:35.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:35 smithi043 ceph-mon[30062]: Deploying daemon mon.c on smithi043 2024-03-21T12:47:35.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:35 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.155:0/2264383897' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T12:47:35.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:35 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:47:35.879 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-21T12:47:35.879 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T12:47:36.127 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T12:47:37.034 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:36 smithi043 systemd[1]: Starting Ceph mon.c for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T12:47:37.099 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:37 smithi155 ceph-mon[33846]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:37.357 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[30062]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:37.357 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 podman[41660]: 2024-03-21 12:47:37.155341766 +0000 UTC m=+0.111432277 container create 52be75e7f9260000c89810f899872047387d423fe663942d4acbbb32974ce509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-21T12:47:37.357 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 podman[41660]: 2024-03-21 12:47:37.061733371 +0000 UTC m=+0.017823896 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93 2024-03-21T12:47:37.358 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 podman[41660]: 2024-03-21 12:47:37.357777075 +0000 UTC m=+0.313867596 container init 52be75e7f9260000c89810f899872047387d423fe663942d4acbbb32974ce509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T12:47:37.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 podman[41660]: 2024-03-21 12:47:37.363459533 +0000 UTC m=+0.319550047 container start 52be75e7f9260000c89810f899872047387d423fe663942d4acbbb32974ce509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-c, ceph=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319) 2024-03-21T12:47:37.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: set uid:gid to 167:167 (ceph:ceph) 2024-03-21T12:47:37.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: ceph version 19.0.0-2267-g960fbbe4 (960fbbe4645c2becc8084669be194c95183c7753) squid (dev), process ceph-mon, pid 7 2024-03-21T12:47:37.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: pidfile_write: ignore empty --pid-file 2024-03-21T12:47:37.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: load: jerasure load: lrc 2024-03-21T12:47:37.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: RocksDB version: 7.9.2 2024-03-21T12:47:37.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Git sha 0 2024-03-21T12:47:37.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Compile date 2024-03-21 08:53:26 2024-03-21T12:47:37.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: DB SUMMARY 2024-03-21T12:47:37.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: DB Session ID: JK613PUI25KUQE8YN0M9 2024-03-21T12:47:37.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: CURRENT file: CURRENT 2024-03-21T12:47:37.699 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: IDENTITY file: IDENTITY 2024-03-21T12:47:37.700 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-03-21T12:47:37.700 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-03-21T12:47:37.700 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 566 ; 2024-03-21T12:47:37.700 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.error_if_exists: 0 2024-03-21T12:47:37.700 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.create_if_missing: 0 2024-03-21T12:47:37.700 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.paranoid_checks: 1 2024-03-21T12:47:37.700 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-21T12:47:37.700 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-21T12:47:37.700 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-21T12:47:37.700 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.env: 0x560ab38d9b20 2024-03-21T12:47:37.700 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.fs: PosixFileSystem 2024-03-21T12:47:37.700 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.info_log: 0x560ab4e21920 2024-03-21T12:47:37.700 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_file_opening_threads: 16 2024-03-21T12:47:37.700 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.statistics: (nil) 2024-03-21T12:47:37.701 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.use_fsync: 0 2024-03-21T12:47:37.701 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_log_file_size: 0 2024-03-21T12:47:37.701 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-21T12:47:37.701 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-21T12:47:37.701 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.keep_log_file_num: 1000 2024-03-21T12:47:37.701 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.recycle_log_file_num: 0 2024-03-21T12:47:37.701 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.allow_fallocate: 1 2024-03-21T12:47:37.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.allow_mmap_reads: 0 2024-03-21T12:47:37.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.allow_mmap_writes: 0 2024-03-21T12:47:37.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.use_direct_reads: 0 2024-03-21T12:47:37.702 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-21T12:47:37.706 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.create_missing_column_families: 0 2024-03-21T12:47:37.706 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.db_log_dir: 2024-03-21T12:47:37.706 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.wal_dir: 2024-03-21T12:47:37.706 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-21T12:47:37.706 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-21T12:47:37.706 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-21T12:47:37.706 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-21T12:47:37.706 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-21T12:47:37.706 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-21T12:47:37.706 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.advise_random_on_open: 1 2024-03-21T12:47:37.706 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.db_write_buffer_size: 0 2024-03-21T12:47:37.706 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.write_buffer_manager: 0x560ab4df21e0 2024-03-21T12:47:37.706 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-21T12:47:37.707 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-21T12:47:37.707 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-21T12:47:37.707 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.rate_limiter: (nil) 2024-03-21T12:47:37.707 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-21T12:47:37.707 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.wal_recovery_mode: 2 2024-03-21T12:47:37.707 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.enable_thread_tracking: 0 2024-03-21T12:47:37.707 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.enable_pipelined_write: 0 2024-03-21T12:47:37.707 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.unordered_write: 0 2024-03-21T12:47:37.707 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-21T12:47:37.707 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-21T12:47:37.707 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-21T12:47:37.707 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-21T12:47:37.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.row_cache: None 2024-03-21T12:47:37.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.wal_filter: None 2024-03-21T12:47:37.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-21T12:47:37.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.allow_ingest_behind: 0 2024-03-21T12:47:37.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.two_write_queues: 0 2024-03-21T12:47:37.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.manual_wal_flush: 0 2024-03-21T12:47:37.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.wal_compression: 0 2024-03-21T12:47:37.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.atomic_flush: 0 2024-03-21T12:47:37.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-21T12:47:37.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-21T12:47:37.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-21T12:47:37.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.log_readahead_size: 0 2024-03-21T12:47:37.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-21T12:47:37.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.best_efforts_recovery: 0 2024-03-21T12:47:37.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-21T12:47:37.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-21T12:47:37.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.allow_data_in_errors: 0 2024-03-21T12:47:37.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.db_host_id: __hostname__ 2024-03-21T12:47:37.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-21T12:47:37.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_background_jobs: 2 2024-03-21T12:47:37.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_background_compactions: -1 2024-03-21T12:47:37.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_subcompactions: 1 2024-03-21T12:47:37.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-21T12:47:37.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-21T12:47:37.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-21T12:47:37.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_total_wal_size: 0 2024-03-21T12:47:37.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-21T12:47:37.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-21T12:47:37.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-21T12:47:37.710 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-21T12:47:37.710 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_open_files: -1 2024-03-21T12:47:37.710 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.bytes_per_sync: 0 2024-03-21T12:47:37.710 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-21T12:47:37.710 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-21T12:47:37.710 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.compaction_readahead_size: 0 2024-03-21T12:47:37.710 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_background_flushes: -1 2024-03-21T12:47:37.710 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Compression algorithms supported: 2024-03-21T12:47:37.710 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-21T12:47:37.710 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: kZSTD supported: 0 2024-03-21T12:47:37.710 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: kXpressCompression supported: 0 2024-03-21T12:47:37.710 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: kLZ4HCCompression supported: 1 2024-03-21T12:47:37.710 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: kZlibCompression supported: 1 2024-03-21T12:47:37.710 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: kSnappyCompression supported: 1 2024-03-21T12:47:37.710 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: kLZ4Compression supported: 1 2024-03-21T12:47:37.711 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: kBZip2Compression supported: 0 2024-03-21T12:47:37.711 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-21T12:47:37.711 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-21T12:47:37.711 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-03-21T12:47:37.711 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-21T12:47:37.711 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-21T12:47:37.711 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.merge_operator: 2024-03-21T12:47:37.711 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.compaction_filter: None 2024-03-21T12:47:37.711 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.compaction_filter_factory: None 2024-03-21T12:47:37.711 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.sst_partitioner_factory: None 2024-03-21T12:47:37.712 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-21T12:47:37.712 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-21T12:47:37.712 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560ab4e21a20) 2024-03-21T12:47:37.712 INFO:journalctl@ceph.mon.c.smithi043.stdout: cache_index_and_filter_blocks: 1 2024-03-21T12:47:37.712 INFO:journalctl@ceph.mon.c.smithi043.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-21T12:47:37.712 INFO:journalctl@ceph.mon.c.smithi043.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-21T12:47:37.712 INFO:journalctl@ceph.mon.c.smithi043.stdout: pin_top_level_index_and_filter: 1 2024-03-21T12:47:37.712 INFO:journalctl@ceph.mon.c.smithi043.stdout: index_type: 0 2024-03-21T12:47:37.712 INFO:journalctl@ceph.mon.c.smithi043.stdout: data_block_index_type: 0 2024-03-21T12:47:37.712 INFO:journalctl@ceph.mon.c.smithi043.stdout: index_shortening: 1 2024-03-21T12:47:37.712 INFO:journalctl@ceph.mon.c.smithi043.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-21T12:47:37.712 INFO:journalctl@ceph.mon.c.smithi043.stdout: checksum: 4 2024-03-21T12:47:37.712 INFO:journalctl@ceph.mon.c.smithi043.stdout: no_block_cache: 0 2024-03-21T12:47:37.712 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_cache: 0x560ab4e25090 2024-03-21T12:47:37.712 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_cache_name: BinnedLRUCache 2024-03-21T12:47:37.712 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_cache_options: 2024-03-21T12:47:37.713 INFO:journalctl@ceph.mon.c.smithi043.stdout: capacity : 536870912 2024-03-21T12:47:37.713 INFO:journalctl@ceph.mon.c.smithi043.stdout: num_shard_bits : 4 2024-03-21T12:47:37.713 INFO:journalctl@ceph.mon.c.smithi043.stdout: strict_capacity_limit : 0 2024-03-21T12:47:37.713 INFO:journalctl@ceph.mon.c.smithi043.stdout: high_pri_pool_ratio: 0.000 2024-03-21T12:47:37.713 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_cache_compressed: (nil) 2024-03-21T12:47:37.713 INFO:journalctl@ceph.mon.c.smithi043.stdout: persistent_cache: (nil) 2024-03-21T12:47:37.713 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_size: 4096 2024-03-21T12:47:37.713 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_size_deviation: 10 2024-03-21T12:47:37.713 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_restart_interval: 16 2024-03-21T12:47:37.713 INFO:journalctl@ceph.mon.c.smithi043.stdout: index_block_restart_interval: 1 2024-03-21T12:47:37.713 INFO:journalctl@ceph.mon.c.smithi043.stdout: metadata_block_size: 4096 2024-03-21T12:47:37.713 INFO:journalctl@ceph.mon.c.smithi043.stdout: partition_filters: 0 2024-03-21T12:47:37.713 INFO:journalctl@ceph.mon.c.smithi043.stdout: use_delta_encoding: 1 2024-03-21T12:47:37.713 INFO:journalctl@ceph.mon.c.smithi043.stdout: filter_policy: bloomfilter 2024-03-21T12:47:37.713 INFO:journalctl@ceph.mon.c.smithi043.stdout: whole_key_filtering: 1 2024-03-21T12:47:37.714 INFO:journalctl@ceph.mon.c.smithi043.stdout: verify_compression: 0 2024-03-21T12:47:37.714 INFO:journalctl@ceph.mon.c.smithi043.stdout: read_amp_bytes_per_bit: 0 2024-03-21T12:47:37.714 INFO:journalctl@ceph.mon.c.smithi043.stdout: format_version: 5 2024-03-21T12:47:37.714 INFO:journalctl@ceph.mon.c.smithi043.stdout: enable_index_compression: 1 2024-03-21T12:47:37.714 INFO:journalctl@ceph.mon.c.smithi043.stdout: block_align: 0 2024-03-21T12:47:37.714 INFO:journalctl@ceph.mon.c.smithi043.stdout: max_auto_readahead_size: 262144 2024-03-21T12:47:37.714 INFO:journalctl@ceph.mon.c.smithi043.stdout: prepopulate_block_cache: 0 2024-03-21T12:47:37.714 INFO:journalctl@ceph.mon.c.smithi043.stdout: initial_auto_readahead_size: 8192 2024-03-21T12:47:37.714 INFO:journalctl@ceph.mon.c.smithi043.stdout: num_file_reads_for_auto_readahead: 2 2024-03-21T12:47:37.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.write_buffer_size: 33554432 2024-03-21T12:47:37.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_write_buffer_number: 2 2024-03-21T12:47:37.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.compression: NoCompression 2024-03-21T12:47:37.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.bottommost_compression: Disabled 2024-03-21T12:47:37.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.prefix_extractor: nullptr 2024-03-21T12:47:37.714 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-21T12:47:37.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.num_levels: 7 2024-03-21T12:47:37.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-21T12:47:37.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-21T12:47:37.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-21T12:47:37.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-21T12:47:37.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-21T12:47:37.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-21T12:47:37.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-21T12:47:37.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-21T12:47:37.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-21T12:47:37.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-21T12:47:37.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-21T12:47:37.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-21T12:47:37.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-21T12:47:37.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.compression_opts.level: 32767 2024-03-21T12:47:37.715 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.compression_opts.strategy: 0 2024-03-21T12:47:37.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-21T12:47:37.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-21T12:47:37.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-21T12:47:37.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-21T12:47:37.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.compression_opts.enabled: false 2024-03-21T12:47:37.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-21T12:47:37.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-21T12:47:37.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-21T12:47:37.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-21T12:47:37.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.target_file_size_base: 67108864 2024-03-21T12:47:37.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-21T12:47:37.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-21T12:47:37.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-21T12:47:37.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-21T12:47:37.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-21T12:47:37.716 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-21T12:47:37.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-21T12:47:37.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-21T12:47:37.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-21T12:47:37.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-21T12:47:37.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-21T12:47:37.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-21T12:47:37.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-21T12:47:37.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-21T12:47:37.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.arena_block_size: 1048576 2024-03-21T12:47:37.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-21T12:47:37.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-21T12:47:37.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.disable_auto_compactions: 0 2024-03-21T12:47:37.717 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-21T12:47:37.718 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-21T12:47:37.718 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-21T12:47:37.718 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-21T12:47:37.718 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-21T12:47:37.718 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-21T12:47:37.718 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-21T12:47:37.718 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-21T12:47:37.718 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-21T12:47:37.718 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-21T12:47:37.718 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.table_properties_collectors: 2024-03-21T12:47:37.718 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.inplace_update_support: 0 2024-03-21T12:47:37.718 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-21T12:47:37.718 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-21T12:47:37.718 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-21T12:47:37.718 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-21T12:47:37.719 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.bloom_locality: 0 2024-03-21T12:47:37.719 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.max_successive_merges: 0 2024-03-21T12:47:37.719 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-21T12:47:37.719 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.paranoid_file_checks: 0 2024-03-21T12:47:37.719 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.force_consistency_checks: 1 2024-03-21T12:47:37.719 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.report_bg_io_stats: 0 2024-03-21T12:47:37.719 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.ttl: 2592000 2024-03-21T12:47:37.719 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-21T12:47:37.719 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-21T12:47:37.719 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-21T12:47:37.719 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.enable_blob_files: false 2024-03-21T12:47:37.719 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.min_blob_size: 0 2024-03-21T12:47:37.719 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.blob_file_size: 268435456 2024-03-21T12:47:37.719 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-21T12:47:37.719 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-21T12:47:37.720 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-21T12:47:37.720 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-21T12:47:37.720 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-21T12:47:37.721 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.blob_file_starting_level: 0 2024-03-21T12:47:37.721 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-21T12:47:37.721 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-21T12:47:37.721 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-21T12:47:37.721 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4d1a2d78-2998-4cc7-9c5f-a503e0f315d8 2024-03-21T12:47:37.721 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711025257393094, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-03-21T12:47:37.721 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-03-21T12:47:37.721 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711025257393554, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1677, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 578, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 456, "raw_average_value_size": 91, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711025257, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4d1a2d78-2998-4cc7-9c5f-a503e0f315d8", "db_session_id": "JK613PUI25KUQE8YN0M9", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-03-21T12:47:37.721 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711025257393646, "job": 1, "event": "recovery_finished"} 2024-03-21T12:47:37.721 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-03-21T12:47:37.722 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-21T12:47:37.722 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560ab4f14000 2024-03-21T12:47:37.722 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: DB pointer 0x560ab4efe000 2024-03-21T12:47:37.722 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-21T12:47:37.722 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-21T12:47:37.722 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** DB Stats ** 2024-03-21T12:47:37.722 INFO:journalctl@ceph.mon.c.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T12:47:37.722 INFO:journalctl@ceph.mon.c.smithi043.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-21T12:47:37.722 INFO:journalctl@ceph.mon.c.smithi043.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T12:47:37.722 INFO:journalctl@ceph.mon.c.smithi043.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T12:47:37.722 INFO:journalctl@ceph.mon.c.smithi043.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-21T12:47:37.722 INFO:journalctl@ceph.mon.c.smithi043.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T12:47:37.722 INFO:journalctl@ceph.mon.c.smithi043.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T12:47:37.722 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-03-21T12:47:37.722 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** Compaction Stats [default] ** 2024-03-21T12:47:37.723 INFO:journalctl@ceph.mon.c.smithi043.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T12:47:37.723 INFO:journalctl@ceph.mon.c.smithi043.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-21T12:47:37.723 INFO:journalctl@ceph.mon.c.smithi043.stdout: L0 1/0 1.64 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T12:47:37.723 INFO:journalctl@ceph.mon.c.smithi043.stdout: Sum 1/0 1.64 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T12:47:37.723 INFO:journalctl@ceph.mon.c.smithi043.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T12:47:37.723 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-03-21T12:47:37.723 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** Compaction Stats [default] ** 2024-03-21T12:47:37.723 INFO:journalctl@ceph.mon.c.smithi043.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T12:47:37.723 INFO:journalctl@ceph.mon.c.smithi043.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-21T12:47:37.723 INFO:journalctl@ceph.mon.c.smithi043.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-03-21T12:47:37.724 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-03-21T12:47:37.724 INFO:journalctl@ceph.mon.c.smithi043.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-21T12:47:37.724 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-03-21T12:47:37.724 INFO:journalctl@ceph.mon.c.smithi043.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T12:47:37.724 INFO:journalctl@ceph.mon.c.smithi043.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-21T12:47:37.724 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-21T12:47:37.724 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-21T12:47:37.724 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-21T12:47:37.724 INFO:journalctl@ceph.mon.c.smithi043.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-21T12:47:37.724 INFO:journalctl@ceph.mon.c.smithi043.stdout: Cumulative compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T12:47:37.724 INFO:journalctl@ceph.mon.c.smithi043.stdout: Interval compaction: 0.00 GB write, 0.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T12:47:37.725 INFO:journalctl@ceph.mon.c.smithi043.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-21T12:47:37.725 INFO:journalctl@ceph.mon.c.smithi043.stdout: Block cache BinnedLRUCache@0x560ab4e25090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-03-21T12:47:37.725 INFO:journalctl@ceph.mon.c.smithi043.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-03-21T12:47:37.725 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-03-21T12:47:37.725 INFO:journalctl@ceph.mon.c.smithi043.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-21T12:47:37.725 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-03-21T12:47:37.725 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: using public_addrv v1:172.21.15.43:6790/0 2024-03-21T12:47:37.725 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: starting mon.c rank -1 at public addrs v1:172.21.15.43:6790/0 at bind addrs v1:172.21.15.43:6790/0 mon_data /var/lib/ceph/mon/ceph-c fsid c06c8d1a-e780-11ee-95cd-87774f69a715 2024-03-21T12:47:37.725 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: mon.c@-1(???) e0 preinit fsid c06c8d1a-e780-11ee-95cd-87774f69a715 2024-03-21T12:47:37.725 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: mon.c@-1(synchronizing).mds e1 new map 2024-03-21T12:47:37.725 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: mon.c@-1(synchronizing).mds e1 print_map 2024-03-21T12:47:37.725 INFO:journalctl@ceph.mon.c.smithi043.stdout: e1 2024-03-21T12:47:37.725 INFO:journalctl@ceph.mon.c.smithi043.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T12:47:37.725 INFO:journalctl@ceph.mon.c.smithi043.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T12:47:37.725 INFO:journalctl@ceph.mon.c.smithi043.stdout: legacy client fscid: -1 2024-03-21T12:47:37.726 INFO:journalctl@ceph.mon.c.smithi043.stdout: 2024-03-21T12:47:37.726 INFO:journalctl@ceph.mon.c.smithi043.stdout: No filesystems configured 2024-03-21T12:47:37.726 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-03-21T12:47:37.726 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-03-21T12:47:37.726 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-03-21T12:47:37.726 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-03-21T12:47:37.726 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-03-21T12:47:37.726 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-03-21T12:47:37.726 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-21T12:47:37.726 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T12:47:37.726 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T12:47:37.726 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T12:47:37.726 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:37.727 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:37.727 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='client.14180 v1:172.21.15.43:0/3150102005' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T12:47:37.727 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:37.727 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:37.727 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2835169065' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-21T12:47:37.727 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:37.727 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2835169065' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-21T12:47:37.727 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T12:47:37.727 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:37.727 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:37.727 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:37.727 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:37.727 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:47:37.727 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:37.727 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:47:37.728 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: Updating smithi155:/etc/ceph/ceph.conf 2024-03-21T12:47:37.728 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: Updating smithi155:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T12:47:37.728 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-03-21T12:47:37.728 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: Updating smithi155:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T12:47:37.728 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:37.728 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:37.728 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:37.728 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:37.728 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:37.728 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:47:37.728 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:37.728 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:47:37.728 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:37.728 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T12:47:37.728 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:37.728 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='client.14184 v1:172.21.15.155:0/3986417470' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi043:[v1:172.21.15.43:6789]=a;smithi043:[v1:172.21.15.43:6790]=c;smithi155:[v1:172.21.15.155:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:47:37.729 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: Saving service mon spec with placement smithi043:[v1:172.21.15.43:6789]=a;smithi043:[v1:172.21.15.43:6790]=c;smithi155:[v1:172.21.15.155:6789]=b;count:3 2024-03-21T12:47:37.729 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: Deploying daemon mon.b on smithi155 2024-03-21T12:47:37.729 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:37.729 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:37.729 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.155:0/2625709698' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T12:47:37.729 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T12:47:37.729 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:47:37.729 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: mon.a calling monitor election 2024-03-21T12:47:37.729 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:37.729 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:47:37.729 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:47:37.729 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: mon.b calling monitor election 2024-03-21T12:47:37.729 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:37.729 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:47:37.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:47:37.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:47:37.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-03-21T12:47:37.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: monmap e2: 2 mons at {a=v1:172.21.15.43:6789/0,b=v1:172.21.15.155:6789/0} removed_ranks: {} disallowed_leaders: {} 2024-03-21T12:47:37.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: fsmap 2024-03-21T12:47:37.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T12:47:37.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: mgrmap e13: y(active, since 42s) 2024-03-21T12:47:37.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: overall HEALTH_OK 2024-03-21T12:47:37.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:37.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:37.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:37.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T12:47:37.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:37.730 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:37.731 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: Deploying daemon mon.c on smithi043 2024-03-21T12:47:37.731 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.155:0/2264383897' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T12:47:37.731 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:47:37.731 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:37.731 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 ceph-mon[41699]: mon.c@-1(synchronizing).paxosservice(auth 1..4) refresh upgraded, format 0 -> 3 2024-03-21T12:47:37.731 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 bash[41660]: 52be75e7f9260000c89810f899872047387d423fe663942d4acbbb32974ce509 2024-03-21T12:47:37.731 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:37 smithi043 systemd[1]: Started Ceph mon.c for c06c8d1a-e780-11ee-95cd-87774f69a715. 2024-03-21T12:47:42.451 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.449+0000 7fde6693c700 1 -- 172.21.15.155:0/585457378 >> v1:172.21.15.43:6789/0 conn(0x7fde4c0035f0 legacy=0x7fde4c005a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:42.451 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.449+0000 7fde6693c700 1 -- 172.21.15.155:0/585457378 shutdown_connections 2024-03-21T12:47:42.451 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.449+0000 7fde6693c700 1 -- 172.21.15.155:0/585457378 >> 172.21.15.155:0/585457378 conn(0x7fde600fc5c0 msgr2=0x7fde600fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:47:42.451 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.449+0000 7fde6693c700 1 -- 172.21.15.155:0/585457378 shutdown_connections 2024-03-21T12:47:42.451 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.449+0000 7fde6693c700 1 -- 172.21.15.155:0/585457378 wait complete. 2024-03-21T12:47:42.451 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.450+0000 7fde6693c700 1 Processor -- start 2024-03-21T12:47:42.451 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.450+0000 7fde6693c700 1 -- start start 2024-03-21T12:47:42.452 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.451+0000 7fde6693c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fde601a6ad0 con 0x7fde60198500 2024-03-21T12:47:42.452 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.451+0000 7fde6693c700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fde601a7cb0 con 0x7fde60198040 2024-03-21T12:47:42.452 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.451+0000 7fde6693c700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fde601a8ef0 con 0x7fde4c0035f0 2024-03-21T12:47:42.452 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.451+0000 7fde5f7fe700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fde60198040 0x7fde601a19b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:55626/0 (socket says 172.21.15.155:55626) 2024-03-21T12:47:42.452 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.451+0000 7fde5f7fe700 1 -- 172.21.15.155:0/2628388387 learned_addr learned my addr 172.21.15.155:0/2628388387 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-03-21T12:47:42.453 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.452+0000 7fde5d7fa700 1 -- 172.21.15.155:0/2628388387 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089481802 0 0) 0x7fde601a6ad0 con 0x7fde60198500 2024-03-21T12:47:42.453 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.452+0000 7fde5d7fa700 1 -- 172.21.15.155:0/2628388387 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde48003bf0 con 0x7fde60198500 2024-03-21T12:47:42.454 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.453+0000 7fde5d7fa700 1 -- 172.21.15.155:0/2628388387 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1048219385 0 0) 0x7fde48003bf0 con 0x7fde60198500 2024-03-21T12:47:42.454 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.453+0000 7fde5d7fa700 1 -- 172.21.15.155:0/2628388387 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fde601a6ad0 con 0x7fde60198500 2024-03-21T12:47:42.454 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.453+0000 7fde5d7fa700 1 -- 172.21.15.155:0/2628388387 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fde58001cd0 con 0x7fde60198500 2024-03-21T12:47:42.454 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.453+0000 7fde5d7fa700 1 -- 172.21.15.155:0/2628388387 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2486428839 0 0) 0x7fde601a6ad0 con 0x7fde60198500 2024-03-21T12:47:42.454 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.453+0000 7fde5d7fa700 1 -- 172.21.15.155:0/2628388387 >> v1:172.21.15.43:6790/0 conn(0x7fde4c0035f0 legacy=0x7fde60197930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:42.454 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.453+0000 7fde5d7fa700 1 -- 172.21.15.155:0/2628388387 >> v1:172.21.15.155:6789/0 conn(0x7fde60198040 legacy=0x7fde601a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:42.454 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.453+0000 7fde5d7fa700 1 -- 172.21.15.155:0/2628388387 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde601aa130 con 0x7fde60198500 2024-03-21T12:47:42.454 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.453+0000 7fde6693c700 1 -- 172.21.15.155:0/2628388387 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde601a9120 con 0x7fde60198500 2024-03-21T12:47:42.454 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.453+0000 7fde6693c700 1 -- 172.21.15.155:0/2628388387 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fde601a9620 con 0x7fde60198500 2024-03-21T12:47:42.454 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.453+0000 7fde5d7fa700 1 -- 172.21.15.155:0/2628388387 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7fde58003f50 con 0x7fde60198500 2024-03-21T12:47:42.455 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.453+0000 7fde5d7fa700 1 -- 172.21.15.155:0/2628388387 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fde58004ed0 con 0x7fde60198500 2024-03-21T12:47:42.455 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.454+0000 7fde5d7fa700 1 -- 172.21.15.155:0/2628388387 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 13) v1 ==== 46315+0+0 (unknown 152183610 0 0) 0x7fde58010590 con 0x7fde60198500 2024-03-21T12:47:42.455 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.454+0000 7fde5d7fa700 1 -- 172.21.15.155:0/2628388387 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (unknown 709340102 0 0) 0x7fde580479c0 con 0x7fde60198500 2024-03-21T12:47:42.455 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.454+0000 7fde6693c700 1 -- 172.21.15.155:0/2628388387 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4c006b70 con 0x7fde60198500 2024-03-21T12:47:42.461 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.460+0000 7fde5d7fa700 1 -- 172.21.15.155:0/2628388387 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7fde58016580 con 0x7fde60198500 2024-03-21T12:47:42.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:42 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T12:47:42.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:42 smithi155 ceph-mon[33846]: mon.a calling monitor election 2024-03-21T12:47:42.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:42 smithi155 ceph-mon[33846]: mon.b calling monitor election 2024-03-21T12:47:42.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:42 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:47:42.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:42 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:47:42.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:42 smithi155 ceph-mon[33846]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:42.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:42 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:47:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:42 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:47:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:42 smithi155 ceph-mon[33846]: mon.c calling monitor election 2024-03-21T12:47:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:42 smithi155 ceph-mon[33846]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:42 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:47:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:42 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:47:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:42 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:47:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:42 smithi155 ceph-mon[33846]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-21T12:47:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:42 smithi155 ceph-mon[33846]: monmap e3: 3 mons at {a=v1:172.21.15.43:6789/0,b=v1:172.21.15.155:6789/0,c=v1:172.21.15.43:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-03-21T12:47:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:42 smithi155 ceph-mon[33846]: fsmap 2024-03-21T12:47:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:42 smithi155 ceph-mon[33846]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T12:47:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:42 smithi155 ceph-mon[33846]: mgrmap e13: y(active, since 50s) 2024-03-21T12:47:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:42 smithi155 ceph-mon[33846]: overall HEALTH_OK 2024-03-21T12:47:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:42 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:42 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:42.745 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.744+0000 7fde6693c700 1 -- 172.21.15.155:0/2628388387 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fde4c007130 con 0x7fde60198500 2024-03-21T12:47:42.745 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.745+0000 7fde5d7fa700 1 -- 172.21.15.155:0/2628388387 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1127 (unknown 409899127 0 1466110967) 0x7fde58015e80 con 0x7fde60198500 2024-03-21T12:47:42.745 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-03-21T12:47:42.746 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":3,"fsid":"c06c8d1a-e780-11ee-95cd-87774f69a715","modified":"2024-03-21T12:47:37.412538Z","created":"2024-03-21T12:45:43.688830Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6789","nonce":0}]},"addr":"172.21.15.43:6789/0","public_addr":"172.21.15.43:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6789","nonce":0}]},"addr":"172.21.15.155:6789/0","public_addr":"172.21.15.155:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6790","nonce":0}]},"addr":"172.21.15.43:6790/0","public_addr":"172.21.15.43:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-03-21T12:47:42.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.747+0000 7fde6693c700 1 -- 172.21.15.155:0/2628388387 >> v1:172.21.15.43:6800/4081968544 conn(0x7fde4803a3d0 legacy=0x7fde4803c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:42.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.747+0000 7fde6693c700 1 -- 172.21.15.155:0/2628388387 >> v1:172.21.15.43:6789/0 conn(0x7fde60198500 legacy=0x7fde601a5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:42.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.747+0000 7fde6693c700 1 -- 172.21.15.155:0/2628388387 shutdown_connections 2024-03-21T12:47:42.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.747+0000 7fde6693c700 1 -- 172.21.15.155:0/2628388387 >> 172.21.15.155:0/2628388387 conn(0x7fde600fc5c0 msgr2=0x7fde600fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:47:42.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.747+0000 7fde6693c700 1 -- 172.21.15.155:0/2628388387 shutdown_connections 2024-03-21T12:47:42.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:42.747+0000 7fde6693c700 1 -- 172.21.15.155:0/2628388387 wait complete. 2024-03-21T12:47:42.748 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 3 2024-03-21T12:47:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T12:47:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[30062]: mon.a calling monitor election 2024-03-21T12:47:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[30062]: mon.b calling monitor election 2024-03-21T12:47:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:47:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:47:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[30062]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:47:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:47:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[30062]: mon.c calling monitor election 2024-03-21T12:47:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[30062]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:47:42.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:47:42.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:47:42.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[30062]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-21T12:47:42.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[30062]: monmap e3: 3 mons at {a=v1:172.21.15.43:6789/0,b=v1:172.21.15.155:6789/0,c=v1:172.21.15.43:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-03-21T12:47:42.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[30062]: fsmap 2024-03-21T12:47:42.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[30062]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T12:47:42.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[30062]: mgrmap e13: y(active, since 50s) 2024-03-21T12:47:42.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[30062]: overall HEALTH_OK 2024-03-21T12:47:42.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:42.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:42.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T12:47:42.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[41699]: mon.a calling monitor election 2024-03-21T12:47:42.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[41699]: mon.b calling monitor election 2024-03-21T12:47:42.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:47:42.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:47:42.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[41699]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:42.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:47:42.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:47:42.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[41699]: mon.c calling monitor election 2024-03-21T12:47:42.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[41699]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:42.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:47:42.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:47:42.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:47:42.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[41699]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-21T12:47:42.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[41699]: monmap e3: 3 mons at {a=v1:172.21.15.43:6789/0,b=v1:172.21.15.155:6789/0,c=v1:172.21.15.43:6790/0} removed_ranks: {} disallowed_leaders: {} 2024-03-21T12:47:42.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[41699]: fsmap 2024-03-21T12:47:42.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[41699]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T12:47:42.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[41699]: mgrmap e13: y(active, since 50s) 2024-03-21T12:47:42.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[41699]: overall HEALTH_OK 2024-03-21T12:47:42.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:42.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:42 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:43.411 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-21T12:47:43.411 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph config generate-minimal-conf 2024-03-21T12:47:43.689 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:43 smithi043 ceph-mon[30062]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:43.689 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:43 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:43.690 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:43 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:43.690 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:43 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:47:43.690 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.155:0/2628388387' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T12:47:43.690 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:43 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:47:43.690 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:43 smithi043 ceph-mon[41699]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:43.690 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:43 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:43.690 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:43 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:43.690 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:43 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:47:43.690 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.155:0/2628388387' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T12:47:43.690 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:43 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:47:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:43 smithi155 ceph-mon[33846]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:43 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:43 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:43 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:47:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.155:0/2628388387' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T12:47:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:43 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:47:44.065 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:47:45.328 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[41699]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:45.328 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.328 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.328 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.328 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.328 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:45.328 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:47:45.328 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[41699]: Updating smithi043:/etc/ceph/ceph.conf 2024-03-21T12:47:45.329 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[41699]: Updating smithi155:/etc/ceph/ceph.conf 2024-03-21T12:47:45.329 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[41699]: Updating smithi043:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T12:47:45.329 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[41699]: Updating smithi155:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T12:47:45.329 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.329 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.329 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.329 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.329 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.329 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.329 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.329 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.329 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.329 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[41699]: Reconfiguring mon.a (unknown last config time)... 2024-03-21T12:47:45.329 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T12:47:45.329 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T12:47:45.330 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:45.330 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[41699]: Reconfiguring daemon mon.a on smithi043 2024-03-21T12:47:45.330 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[30062]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:45.330 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.330 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.330 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.330 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.330 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:45.330 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:47:45.330 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[30062]: Updating smithi043:/etc/ceph/ceph.conf 2024-03-21T12:47:45.330 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[30062]: Updating smithi155:/etc/ceph/ceph.conf 2024-03-21T12:47:45.331 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[30062]: Updating smithi043:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T12:47:45.331 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[30062]: Updating smithi155:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T12:47:45.331 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.331 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.331 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.331 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.331 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.331 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.332 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.332 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.332 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.332 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[30062]: Reconfiguring mon.a (unknown last config time)... 2024-03-21T12:47:45.332 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T12:47:45.332 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T12:47:45.332 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:45.332 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:45 smithi043 ceph-mon[30062]: Reconfiguring daemon mon.a on smithi043 2024-03-21T12:47:45.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:45 smithi155 ceph-mon[33846]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:45.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:45 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:45 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:45 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:45 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:45 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:45.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:45 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:47:45.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:45 smithi155 ceph-mon[33846]: Updating smithi043:/etc/ceph/ceph.conf 2024-03-21T12:47:45.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:45 smithi155 ceph-mon[33846]: Updating smithi155:/etc/ceph/ceph.conf 2024-03-21T12:47:45.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:45 smithi155 ceph-mon[33846]: Updating smithi043:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T12:47:45.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:45 smithi155 ceph-mon[33846]: Updating smithi155:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T12:47:45.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:45 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:45 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:45 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:45 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:45 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:45 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:45 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:45 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:45 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:45.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:45 smithi155 ceph-mon[33846]: Reconfiguring mon.a (unknown last config time)... 2024-03-21T12:47:45.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:45 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T12:47:45.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:45 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T12:47:45.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:45 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:45.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:45 smithi155 ceph-mon[33846]: Reconfiguring daemon mon.a on smithi043 2024-03-21T12:47:46.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.382+0000 7ff2e4aff700 1 -- 172.21.15.43:0/2489902590 >> v1:172.21.15.43:6789/0 conn(0x7ff2e0107a90 legacy=0x7ff2e0107e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:46.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.382+0000 7ff2e4aff700 1 -- 172.21.15.43:0/2489902590 shutdown_connections 2024-03-21T12:47:46.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.382+0000 7ff2e4aff700 1 -- 172.21.15.43:0/2489902590 >> 172.21.15.43:0/2489902590 conn(0x7ff2e00fd580 msgr2=0x7ff2e00ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:47:46.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.382+0000 7ff2e4aff700 1 -- 172.21.15.43:0/2489902590 shutdown_connections 2024-03-21T12:47:46.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.382+0000 7ff2e4aff700 1 -- 172.21.15.43:0/2489902590 wait complete. 2024-03-21T12:47:46.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.383+0000 7ff2e4aff700 1 Processor -- start 2024-03-21T12:47:46.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.383+0000 7ff2e4aff700 1 -- start start 2024-03-21T12:47:46.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.383+0000 7ff2e4aff700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff2e01a6b50 con 0x7ff2e019cf70 2024-03-21T12:47:46.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.383+0000 7ff2e4aff700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff2e01a7d90 con 0x7ff2e01a2e40 2024-03-21T12:47:46.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.383+0000 7ff2e4aff700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff2e01a8fd0 con 0x7ff2e0107a90 2024-03-21T12:47:46.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.383+0000 7ff2ddd9b700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff2e019cf70 0x7ff2e01a1720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37850/0 (socket says 172.21.15.43:37850) 2024-03-21T12:47:46.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.383+0000 7ff2ddd9b700 1 -- 172.21.15.43:0/1040201902 learned_addr learned my addr 172.21.15.43:0/1040201902 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:47:46.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.383+0000 7ff2de59c700 1 --1- 172.21.15.43:0/1040201902 >> v1:172.21.15.43:6790/0 conn(0x7ff2e0107a90 0x7ff2e019c860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:54536/0 (socket says 172.21.15.43:54536) 2024-03-21T12:47:46.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.384+0000 7ff2cf7fe700 1 -- 172.21.15.43:0/1040201902 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2655269294 0 0) 0x7ff2e01a6b50 con 0x7ff2e019cf70 2024-03-21T12:47:46.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.384+0000 7ff2cf7fe700 1 -- 172.21.15.43:0/1040201902 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2c8003bf0 con 0x7ff2e019cf70 2024-03-21T12:47:46.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.384+0000 7ff2cf7fe700 1 -- 172.21.15.43:0/1040201902 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1129578619 0 0) 0x7ff2e01a8fd0 con 0x7ff2e0107a90 2024-03-21T12:47:46.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.384+0000 7ff2cf7fe700 1 -- 172.21.15.43:0/1040201902 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2e01a6b50 con 0x7ff2e0107a90 2024-03-21T12:47:46.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.384+0000 7ff2cf7fe700 1 -- 172.21.15.43:0/1040201902 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3195662674 0 0) 0x7ff2c8003bf0 con 0x7ff2e019cf70 2024-03-21T12:47:46.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.384+0000 7ff2cf7fe700 1 -- 172.21.15.43:0/1040201902 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff2e01a8fd0 con 0x7ff2e019cf70 2024-03-21T12:47:46.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.384+0000 7ff2cf7fe700 1 -- 172.21.15.43:0/1040201902 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7ff2d0001760 con 0x7ff2e019cf70 2024-03-21T12:47:46.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.384+0000 7ff2cf7fe700 1 -- 172.21.15.43:0/1040201902 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2715760997 0 0) 0x7ff2e01a7d90 con 0x7ff2e01a2e40 2024-03-21T12:47:46.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.384+0000 7ff2cf7fe700 1 -- 172.21.15.43:0/1040201902 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2c8003bf0 con 0x7ff2e01a2e40 2024-03-21T12:47:46.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.384+0000 7ff2cf7fe700 1 -- 172.21.15.43:0/1040201902 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3129308305 0 0) 0x7ff2e01a6b50 con 0x7ff2e0107a90 2024-03-21T12:47:46.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.384+0000 7ff2cf7fe700 1 -- 172.21.15.43:0/1040201902 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff2e01a7d90 con 0x7ff2e0107a90 2024-03-21T12:47:46.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.384+0000 7ff2cf7fe700 1 -- 172.21.15.43:0/1040201902 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 403618260 0 0) 0x7ff2e01a8fd0 con 0x7ff2e019cf70 2024-03-21T12:47:46.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.384+0000 7ff2cf7fe700 1 -- 172.21.15.43:0/1040201902 >> v1:172.21.15.43:6790/0 conn(0x7ff2e0107a90 legacy=0x7ff2e019c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:46.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.384+0000 7ff2cf7fe700 1 -- 172.21.15.43:0/1040201902 >> v1:172.21.15.155:6789/0 conn(0x7ff2e01a2e40 legacy=0x7ff2e01a5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:46.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.384+0000 7ff2cf7fe700 1 -- 172.21.15.43:0/1040201902 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e01aa210 con 0x7ff2e019cf70 2024-03-21T12:47:46.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.384+0000 7ff2e4aff700 1 -- 172.21.15.43:0/1040201902 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e01a6d80 con 0x7ff2e019cf70 2024-03-21T12:47:46.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.384+0000 7ff2e4aff700 1 -- 172.21.15.43:0/1040201902 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e01a7240 con 0x7ff2e019cf70 2024-03-21T12:47:46.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.384+0000 7ff2cf7fe700 1 -- 172.21.15.43:0/1040201902 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7ff2d0004dc0 con 0x7ff2e019cf70 2024-03-21T12:47:46.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.384+0000 7ff2cf7fe700 1 -- 172.21.15.43:0/1040201902 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7ff2d0005d10 con 0x7ff2e019cf70 2024-03-21T12:47:46.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.385+0000 7ff2cf7fe700 1 -- 172.21.15.43:0/1040201902 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 13) v1 ==== 46315+0+0 (unknown 152183610 0 0) 0x7ff2d0011450 con 0x7ff2e019cf70 2024-03-21T12:47:46.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.385+0000 7ff2e4aff700 1 -- 172.21.15.43:0/1040201902 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2c0005120 con 0x7ff2e019cf70 2024-03-21T12:47:46.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.385+0000 7ff2cf7fe700 1 -- 172.21.15.43:0/1040201902 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (unknown 709340102 0 0) 0x7ff2d00478a0 con 0x7ff2e019cf70 2024-03-21T12:47:46.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.390+0000 7ff2cf7fe700 1 -- 172.21.15.43:0/1040201902 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7ff2d0017450 con 0x7ff2e019cf70 2024-03-21T12:47:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.596+0000 7ff2e4aff700 1 -- 172.21.15.43:0/1040201902 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7ff2c0005410 con 0x7ff2e019cf70 2024-03-21T12:47:46.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.596+0000 7ff2cf7fe700 1 -- 172.21.15.43:0/1040201902 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+191 (unknown 2189217731 0 1815849672) 0x7ff2d0016d50 con 0x7ff2e019cf70 2024-03-21T12:47:46.597 INFO:teuthology.orchestra.run.smithi043.stdout:# minimal ceph.conf for c06c8d1a-e780-11ee-95cd-87774f69a715 2024-03-21T12:47:46.597 INFO:teuthology.orchestra.run.smithi043.stdout:[global] 2024-03-21T12:47:46.597 INFO:teuthology.orchestra.run.smithi043.stdout: fsid = c06c8d1a-e780-11ee-95cd-87774f69a715 2024-03-21T12:47:46.597 INFO:teuthology.orchestra.run.smithi043.stdout: mon_host = 172.21.15.43:6789/0 172.21.15.155:6789/0 v1:172.21.15.43:6790/0 2024-03-21T12:47:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.598+0000 7ff2e4aff700 1 -- 172.21.15.43:0/1040201902 >> v1:172.21.15.43:6800/4081968544 conn(0x7ff2c803a4f0 legacy=0x7ff2c803c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.598+0000 7ff2e4aff700 1 -- 172.21.15.43:0/1040201902 >> v1:172.21.15.43:6789/0 conn(0x7ff2e019cf70 legacy=0x7ff2e01a1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:46.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.599+0000 7ff2e4aff700 1 -- 172.21.15.43:0/1040201902 shutdown_connections 2024-03-21T12:47:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.599+0000 7ff2e4aff700 1 -- 172.21.15.43:0/1040201902 >> 172.21.15.43:0/1040201902 conn(0x7ff2e00fd580 msgr2=0x7ff2e0109d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:47:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.599+0000 7ff2e4aff700 1 -- 172.21.15.43:0/1040201902 shutdown_connections 2024-03-21T12:47:46.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:46.599+0000 7ff2e4aff700 1 -- 172.21.15.43:0/1040201902 wait complete. 2024-03-21T12:47:47.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:47 smithi043 ceph-mon[30062]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:47.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1040201902' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:47.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:47 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:47 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:47 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:47 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T12:47:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:47 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T12:47:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:47 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:47.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:47 smithi043 ceph-mon[41699]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:47.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1040201902' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:47.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:47 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:47.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:47 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:47.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:47 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:47.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:47 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T12:47:47.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:47 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T12:47:47.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:47 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:47.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:47 smithi155 ceph-mon[33846]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:47.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1040201902' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:47.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:47 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:47.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:47 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:47.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:47 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:47.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:47 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T12:47:47.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:47 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T12:47:47.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:47 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:47.510 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-21T12:47:47.511 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-21T12:47:47.511 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.conf 2024-03-21T12:47:47.541 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-21T12:47:47.541 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T12:47:47.608 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-21T12:47:47.608 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.conf 2024-03-21T12:47:47.644 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-21T12:47:47.644 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T12:47:47.714 INFO:tasks.cephadm:Adding mgr.y on smithi043 2024-03-21T12:47:47.714 INFO:tasks.cephadm:Adding mgr.x on smithi155 2024-03-21T12:47:47.714 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph orch apply mgr '2;smithi043=y;smithi155=x' 2024-03-21T12:47:47.989 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T12:47:48.350 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:48 smithi043 ceph-mon[30062]: Reconfiguring mon.c (monmap changed)... 2024-03-21T12:47:48.350 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:48 smithi043 ceph-mon[30062]: Reconfiguring daemon mon.c on smithi043 2024-03-21T12:47:48.350 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:48 smithi043 ceph-mon[41699]: Reconfiguring mon.c (monmap changed)... 2024-03-21T12:47:48.351 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:48 smithi043 ceph-mon[41699]: Reconfiguring daemon mon.c on smithi043 2024-03-21T12:47:48.361 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:48 smithi155 ceph-mon[33846]: Reconfiguring mon.c (monmap changed)... 2024-03-21T12:47:48.362 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:48 smithi155 ceph-mon[33846]: Reconfiguring daemon mon.c on smithi043 2024-03-21T12:47:49.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.516+0000 7f1d6ee0a700 1 -- 172.21.15.155:0/2096670082 >> v1:172.21.15.43:6789/0 conn(0x7f1d68101c50 legacy=0x7f1d68102070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:49.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.516+0000 7f1d6ee0a700 1 -- 172.21.15.155:0/2096670082 shutdown_connections 2024-03-21T12:47:49.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.516+0000 7f1d6ee0a700 1 -- 172.21.15.155:0/2096670082 >> 172.21.15.155:0/2096670082 conn(0x7f1d680fd1b0 msgr2=0x7f1d680ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:47:49.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.516+0000 7f1d6ee0a700 1 -- 172.21.15.155:0/2096670082 shutdown_connections 2024-03-21T12:47:49.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.516+0000 7f1d6ee0a700 1 -- 172.21.15.155:0/2096670082 wait complete. 2024-03-21T12:47:49.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.516+0000 7f1d6ee0a700 1 Processor -- start 2024-03-21T12:47:49.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.516+0000 7f1d6ee0a700 1 -- start start 2024-03-21T12:47:49.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.516+0000 7f1d6ee0a700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1d681a1e80 con 0x7f1d68101c50 2024-03-21T12:47:49.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.516+0000 7f1d6ee0a700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1d681a30a0 con 0x7f1d68196d20 2024-03-21T12:47:49.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.516+0000 7f1d6ee0a700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1d681a42e0 con 0x7f1d6819e210 2024-03-21T12:47:49.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.517+0000 7f1d67fff700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f1d68196d20 0x7f1d6819caf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:55636/0 (socket says 172.21.15.155:55636) 2024-03-21T12:47:49.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.517+0000 7f1d6cba6700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1d68101c50 0x7f1d68196610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.155:45650/0 (socket says 172.21.15.155:45650) 2024-03-21T12:47:49.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.517+0000 7f1d67fff700 1 -- 172.21.15.155:0/1244425480 learned_addr learned my addr 172.21.15.155:0/1244425480 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-03-21T12:47:49.520 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.517+0000 7f1d65ffb700 1 -- 172.21.15.155:0/1244425480 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1002849981 0 0) 0x7f1d681a30a0 con 0x7f1d68196d20 2024-03-21T12:47:49.520 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.517+0000 7f1d65ffb700 1 -- 172.21.15.155:0/1244425480 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d58003bf0 con 0x7f1d68196d20 2024-03-21T12:47:49.520 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.517+0000 7f1d65ffb700 1 -- 172.21.15.155:0/1244425480 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 861658061 0 0) 0x7f1d681a1e80 con 0x7f1d68101c50 2024-03-21T12:47:49.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.517+0000 7f1d65ffb700 1 -- 172.21.15.155:0/1244425480 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d681a30a0 con 0x7f1d68101c50 2024-03-21T12:47:49.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.517+0000 7f1d65ffb700 1 -- 172.21.15.155:0/1244425480 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 100011774 0 0) 0x7f1d681a42e0 con 0x7f1d6819e210 2024-03-21T12:47:49.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.517+0000 7f1d65ffb700 1 -- 172.21.15.155:0/1244425480 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d681a1e80 con 0x7f1d6819e210 2024-03-21T12:47:49.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.517+0000 7f1d65ffb700 1 -- 172.21.15.155:0/1244425480 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3575155947 0 0) 0x7f1d58003bf0 con 0x7f1d68196d20 2024-03-21T12:47:49.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.517+0000 7f1d65ffb700 1 -- 172.21.15.155:0/1244425480 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1d681a42e0 con 0x7f1d68196d20 2024-03-21T12:47:49.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.517+0000 7f1d65ffb700 1 -- 172.21.15.155:0/1244425480 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f1d54001830 con 0x7f1d68196d20 2024-03-21T12:47:49.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.517+0000 7f1d65ffb700 1 -- 172.21.15.155:0/1244425480 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1948782128 0 0) 0x7f1d681a30a0 con 0x7f1d68101c50 2024-03-21T12:47:49.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.517+0000 7f1d65ffb700 1 -- 172.21.15.155:0/1244425480 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1d58003bf0 con 0x7f1d68101c50 2024-03-21T12:47:49.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.517+0000 7f1d65ffb700 1 -- 172.21.15.155:0/1244425480 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f1d5c003910 con 0x7f1d68101c50 2024-03-21T12:47:49.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.517+0000 7f1d65ffb700 1 -- 172.21.15.155:0/1244425480 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 360444302 0 0) 0x7f1d681a1e80 con 0x7f1d6819e210 2024-03-21T12:47:49.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.517+0000 7f1d65ffb700 1 -- 172.21.15.155:0/1244425480 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1d681a30a0 con 0x7f1d6819e210 2024-03-21T12:47:49.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.517+0000 7f1d65ffb700 1 -- 172.21.15.155:0/1244425480 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f1d60001b50 con 0x7f1d6819e210 2024-03-21T12:47:49.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.517+0000 7f1d65ffb700 1 -- 172.21.15.155:0/1244425480 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3516149829 0 0) 0x7f1d681a42e0 con 0x7f1d68196d20 2024-03-21T12:47:49.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.517+0000 7f1d65ffb700 1 -- 172.21.15.155:0/1244425480 >> v1:172.21.15.43:6790/0 conn(0x7f1d6819e210 legacy=0x7f1d681a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:49.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.517+0000 7f1d65ffb700 1 -- 172.21.15.155:0/1244425480 >> v1:172.21.15.43:6789/0 conn(0x7f1d68101c50 legacy=0x7f1d68196610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:49.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.517+0000 7f1d65ffb700 1 -- 172.21.15.155:0/1244425480 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d681a5520 con 0x7f1d68196d20 2024-03-21T12:47:49.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.517+0000 7f1d6ee0a700 1 -- 172.21.15.155:0/1244425480 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d681a4510 con 0x7f1d68196d20 2024-03-21T12:47:49.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.518+0000 7f1d65ffb700 1 -- 172.21.15.155:0/1244425480 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f1d54004dc0 con 0x7f1d68196d20 2024-03-21T12:47:49.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.518+0000 7f1d65ffb700 1 -- 172.21.15.155:0/1244425480 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f1d54005d80 con 0x7f1d68196d20 2024-03-21T12:47:49.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.518+0000 7f1d6ee0a700 1 -- 172.21.15.155:0/1244425480 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d681a4a50 con 0x7f1d68196d20 2024-03-21T12:47:49.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.518+0000 7f1d65ffb700 1 -- 172.21.15.155:0/1244425480 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 13) v1 ==== 46315+0+0 (unknown 152183610 0 0) 0x7f1d54011280 con 0x7f1d68196d20 2024-03-21T12:47:49.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.519+0000 7f1d65ffb700 1 -- 172.21.15.155:0/1244425480 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (unknown 709340102 0 0) 0x7f1d54047780 con 0x7f1d68196d20 2024-03-21T12:47:49.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.519+0000 7f1d6ee0a700 1 -- 172.21.15.155:0/1244425480 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d680637c0 con 0x7f1d68196d20 2024-03-21T12:47:49.524 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.524+0000 7f1d65ffb700 1 -- 172.21.15.155:0/1244425480 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f1d540171f0 con 0x7f1d68196d20 2024-03-21T12:47:49.738 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.737+0000 7f1d6ee0a700 1 -- 172.21.15.155:0/1244425480 --> v1:172.21.15.43:6800/4081968544 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi043=y;smithi155=x", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d68108390 con 0x7f1d5803a730 2024-03-21T12:47:49.744 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.743+0000 7f1d65ffb700 1 -- 172.21.15.155:0/1244425480 <== mgr.14152 v1:172.21.15.43:6800/4081968544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 325935098) 0x7f1d68108390 con 0x7f1d5803a730 2024-03-21T12:47:49.744 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled mgr update... 2024-03-21T12:47:49.746 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.746+0000 7f1d6ee0a700 1 -- 172.21.15.155:0/1244425480 >> v1:172.21.15.43:6800/4081968544 conn(0x7f1d5803a730 legacy=0x7f1d5803cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:49.746 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.746+0000 7f1d6ee0a700 1 -- 172.21.15.155:0/1244425480 >> v1:172.21.15.155:6789/0 conn(0x7f1d68196d20 legacy=0x7f1d6819caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:49.747 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.746+0000 7f1d6ee0a700 1 -- 172.21.15.155:0/1244425480 shutdown_connections 2024-03-21T12:47:49.747 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.746+0000 7f1d6ee0a700 1 -- 172.21.15.155:0/1244425480 >> 172.21.15.155:0/1244425480 conn(0x7f1d680fd1b0 msgr2=0x7f1d680ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:47:49.747 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.746+0000 7f1d6ee0a700 1 -- 172.21.15.155:0/1244425480 shutdown_connections 2024-03-21T12:47:49.747 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:47:49.746+0000 7f1d6ee0a700 1 -- 172.21.15.155:0/1244425480 wait complete. 2024-03-21T12:47:49.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:49 smithi155 ceph-mon[33846]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:49.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:49 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:49.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:49 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:49.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:49 smithi155 ceph-mon[33846]: Reconfiguring mon.b (monmap changed)... 2024-03-21T12:47:49.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:49 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T12:47:49.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:49 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T12:47:49.759 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:49 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:49.759 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:49 smithi155 ceph-mon[33846]: Reconfiguring daemon mon.b on smithi155 2024-03-21T12:47:49.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:49 smithi043 ceph-mon[30062]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:49.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:49 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:49.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:49 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:49.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:49 smithi043 ceph-mon[30062]: Reconfiguring mon.b (monmap changed)... 2024-03-21T12:47:49.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:49 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T12:47:49.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:49 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T12:47:49.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:49 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:49.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:49 smithi043 ceph-mon[30062]: Reconfiguring daemon mon.b on smithi155 2024-03-21T12:47:49.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:49 smithi043 ceph-mon[41699]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:49.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:49 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:49.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:49 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:49.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:49 smithi043 ceph-mon[41699]: Reconfiguring mon.b (monmap changed)... 2024-03-21T12:47:49.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:49 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T12:47:49.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:49 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T12:47:49.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:49 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:49.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:49 smithi043 ceph-mon[41699]: Reconfiguring daemon mon.b on smithi155 2024-03-21T12:47:50.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:50 smithi155 ceph-mon[33846]: from='client.24103 v1:172.21.15.155:0/1244425480' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi043=y;smithi155=x", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:47:50.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:50 smithi155 ceph-mon[33846]: Saving service mgr spec with placement smithi043=y;smithi155=x;count:2 2024-03-21T12:47:50.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:50 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:50.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:50 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:50.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:50 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:51.188 DEBUG:teuthology.orchestra.run.smithi155:mgr.x> sudo journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mgr.x.service 2024-03-21T12:47:51.190 INFO:tasks.cephadm:Deploying OSDs... 2024-03-21T12:47:51.190 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-21T12:47:51.191 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/scratch_devs of=/dev/stdout 2024-03-21T12:47:51.209 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-21T12:47:51.210 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_1 2024-03-21T12:47:51.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:50 smithi043 ceph-mon[30062]: from='client.24103 v1:172.21.15.155:0/1244425480' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi043=y;smithi155=x", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:47:51.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:50 smithi043 ceph-mon[30062]: Saving service mgr spec with placement smithi043=y;smithi155=x;count:2 2024-03-21T12:47:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:50 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:50 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:50 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:50 smithi043 ceph-mon[41699]: from='client.24103 v1:172.21.15.155:0/1244425480' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi043=y;smithi155=x", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:47:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:50 smithi043 ceph-mon[41699]: Saving service mgr spec with placement smithi043=y;smithi155=x;count:2 2024-03-21T12:47:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:50 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:50 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:50 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:51.251 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-21T12:47:51.251 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T12:47:51.251 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-03-21T12:47:51.251 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T12:47:51.251 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T12:47:51.251 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-03-21 12:47:36.882243204 +0000 2024-03-21T12:47:51.251 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-03-21 12:41:12.885471296 +0000 2024-03-21T12:47:51.251 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-03-21 12:41:12.885471296 +0000 2024-03-21T12:47:51.251 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-03-21 12:41:12.885471296 +0000 2024-03-21T12:47:51.252 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-21T12:47:51.319 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-03-21T12:47:51.319 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-03-21T12:47:51.319 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000120606 s, 4.2 MB/s 2024-03-21T12:47:51.321 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-21T12:47:51.378 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_2 2024-03-21T12:47:51.434 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-21T12:47:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T12:47:51.435 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-03-21T12:47:51.435 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T12:47:51.435 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T12:47:51.435 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-03-21 12:47:36.883243191 +0000 2024-03-21T12:47:51.435 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-03-21 12:41:13.222466994 +0000 2024-03-21T12:47:51.435 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-03-21 12:41:13.222466994 +0000 2024-03-21T12:47:51.435 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-03-21 12:41:13.222466994 +0000 2024-03-21T12:47:51.435 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-21T12:47:51.502 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-03-21T12:47:51.502 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-03-21T12:47:51.502 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000140943 s, 3.6 MB/s 2024-03-21T12:47:51.503 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-21T12:47:51.560 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_3 2024-03-21T12:47:51.618 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-21T12:47:51.619 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T12:47:51.619 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-21T12:47:51.619 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T12:47:51.619 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T12:47:51.619 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-03-21 12:47:36.883243191 +0000 2024-03-21T12:47:51.619 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-03-21 12:41:13.545462871 +0000 2024-03-21T12:47:51.619 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-03-21 12:41:13.545462871 +0000 2024-03-21T12:47:51.619 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-03-21 12:41:13.545462871 +0000 2024-03-21T12:47:51.619 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-21T12:47:51.685 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-03-21T12:47:51.685 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-03-21T12:47:51.685 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000106798 s, 4.8 MB/s 2024-03-21T12:47:51.686 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-21T12:47:51.743 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_4 2024-03-21T12:47:51.798 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-21T12:47:51.798 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T12:47:51.798 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-03-21T12:47:51.798 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T12:47:51.798 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T12:47:51.798 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-03-21 12:47:36.883243191 +0000 2024-03-21T12:47:51.798 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-03-21 12:41:13.891458454 +0000 2024-03-21T12:47:51.798 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-03-21 12:41:13.891458454 +0000 2024-03-21T12:47:51.798 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-03-21 12:41:13.891458454 +0000 2024-03-21T12:47:51.799 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-21T12:47:51.864 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-03-21T12:47:51.864 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-03-21T12:47:51.864 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000105335 s, 4.9 MB/s 2024-03-21T12:47:51.865 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-21T12:47:51.921 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-21T12:47:51.921 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/scratch_devs of=/dev/stdout 2024-03-21T12:47:51.937 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-21T12:47:51.937 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_1 2024-03-21T12:47:51.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:51 smithi043 ceph-mon[30062]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:51 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:47:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:51 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:51 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:47:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:51 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:51 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T12:47:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:51 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T12:47:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:51 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T12:47:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:51 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:51 smithi043 ceph-mon[30062]: Deploying daemon mgr.x on smithi155 2024-03-21T12:47:51.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:51 smithi043 ceph-mon[41699]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:51.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:51 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:47:51.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:51 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:51.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:51 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:47:51.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:51 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:51.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:51 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T12:47:51.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:51 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T12:47:51.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:51 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T12:47:51.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:51 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:51.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:51 smithi043 ceph-mon[41699]: Deploying daemon mgr.x on smithi155 2024-03-21T12:47:51.992 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-21T12:47:51.993 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T12:47:51.993 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-03-21T12:47:51.993 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T12:47:51.993 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T12:47:51.993 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-03-21 12:47:28.437159517 +0000 2024-03-21T12:47:51.993 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-03-21 12:41:11.427234713 +0000 2024-03-21T12:47:51.993 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-03-21 12:41:11.427234713 +0000 2024-03-21T12:47:51.993 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-03-21 12:41:11.427234713 +0000 2024-03-21T12:47:51.993 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-21T12:47:52.057 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:51 smithi155 ceph-mon[33846]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:52.057 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:51 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:47:52.057 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:51 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:52.057 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:51 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:47:52.057 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:51 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:52.057 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:51 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T12:47:52.058 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:51 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T12:47:52.058 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:51 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T12:47:52.058 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:51 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:52.058 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:51 smithi155 ceph-mon[33846]: Deploying daemon mgr.x on smithi155 2024-03-21T12:47:52.059 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-03-21T12:47:52.059 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-03-21T12:47:52.059 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000206714 s, 2.5 MB/s 2024-03-21T12:47:52.060 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-21T12:47:52.116 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_2 2024-03-21T12:47:52.171 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-21T12:47:52.171 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T12:47:52.172 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-03-21T12:47:52.172 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T12:47:52.172 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T12:47:52.172 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-03-21 12:47:28.438159497 +0000 2024-03-21T12:47:52.172 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-03-21 12:41:11.784227725 +0000 2024-03-21T12:47:52.172 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-03-21 12:41:11.784227725 +0000 2024-03-21T12:47:52.172 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-03-21 12:41:11.784227725 +0000 2024-03-21T12:47:52.172 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-21T12:47:52.237 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-03-21T12:47:52.237 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-03-21T12:47:52.237 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000231455 s, 2.2 MB/s 2024-03-21T12:47:52.238 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-21T12:47:52.294 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_3 2024-03-21T12:47:52.355 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-21T12:47:52.355 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T12:47:52.355 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-21T12:47:52.355 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T12:47:52.355 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T12:47:52.355 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-03-21 12:47:28.438159497 +0000 2024-03-21T12:47:52.355 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-03-21 12:41:12.124221071 +0000 2024-03-21T12:47:52.356 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-03-21 12:41:12.124221071 +0000 2024-03-21T12:47:52.356 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-03-21 12:41:12.124221071 +0000 2024-03-21T12:47:52.356 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-21T12:47:52.459 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-03-21T12:47:52.460 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-03-21T12:47:52.460 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000214806 s, 2.4 MB/s 2024-03-21T12:47:52.461 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-21T12:47:52.476 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_4 2024-03-21T12:47:52.535 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-21T12:47:52.535 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T12:47:52.535 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-03-21T12:47:52.535 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T12:47:52.535 INFO:teuthology.orchestra.run.smithi155.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T12:47:52.535 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-03-21 12:47:52.414684514 +0000 2024-03-21T12:47:52.535 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-03-21 12:41:12.470214298 +0000 2024-03-21T12:47:52.535 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-03-21 12:41:12.470214298 +0000 2024-03-21T12:47:52.535 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: 2024-03-21 12:41:12.470214298 +0000 2024-03-21T12:47:52.536 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-21T12:47:52.648 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-03-21T12:47:52.648 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-03-21T12:47:52.648 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000176951 s, 2.9 MB/s 2024-03-21T12:47:52.650 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-21T12:47:52.665 INFO:tasks.cephadm:Deploying osd.0 on smithi043 with /dev/vg_nvme/lv_4... 2024-03-21T12:47:52.665 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-21T12:47:52.707 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:52 smithi155 systemd[1]: Starting Ceph mgr.x for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T12:47:52.915 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:47:53.163 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:52 smithi155 podman[35908]: 2024-03-21 12:47:52.786410078 +0000 UTC m=+0.017638464 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93 2024-03-21T12:47:53.163 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:52 smithi155 podman[35908]: 2024-03-21 12:47:52.89399426 +0000 UTC m=+0.125222700 container create df012bc317847b9b083d81f4a135fb3c3eb88130ccd4ed684696e3d6143d6860 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319) 2024-03-21T12:47:53.163 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:53 smithi155 podman[35908]: 2024-03-21 12:47:53.087993471 +0000 UTC m=+0.319221861 container init df012bc317847b9b083d81f4a135fb3c3eb88130ccd4ed684696e3d6143d6860 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-03-21T12:47:53.163 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:53 smithi155 podman[35908]: 2024-03-21 12:47:53.095080802 +0000 UTC m=+0.326309186 container start df012bc317847b9b083d81f4a135fb3c3eb88130ccd4ed684696e3d6143d6860 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD) 2024-03-21T12:47:53.163 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:53 smithi155 bash[35908]: df012bc317847b9b083d81f4a135fb3c3eb88130ccd4ed684696e3d6143d6860 2024-03-21T12:47:53.457 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:53 smithi155 systemd[1]: Started Ceph mgr.x for c06c8d1a-e780-11ee-95cd-87774f69a715. 2024-03-21T12:47:53.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:53 smithi155 ceph-mon[33846]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:53.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:53 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:53.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:53 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:53.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:53 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:53.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:53 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:53.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:53 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:47:53.958 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:53 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:47:53.658+0000 7f9555e9a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T12:47:53.958 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:53 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:47:53.756+0000 7f9555e9a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T12:47:54.121 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:53 smithi043 ceph-mon[41699]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:54.121 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:53 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:54.122 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:53 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:54.122 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:53 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:54.122 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:53 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:54.122 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:53 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:47:54.122 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:53 smithi043 ceph-mon[30062]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:54.122 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:53 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:54.122 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:53 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:54.123 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:53 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:54.123 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:53 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:54.123 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:53 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:47:54.799 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:54 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:47:54.458+0000 7f9555e9a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T12:47:54.799 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:54 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:47:54.557+0000 7f9555e9a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T12:47:54.799 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:54 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:47:54.702+0000 7f9555e9a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T12:47:54.799 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:54 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:47:54.798+0000 7f9555e9a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T12:47:55.058 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-21T12:47:55.058 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-21T12:47:55.059 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-03-21T12:47:55.059 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-03-21T12:47:55.059 INFO:teuthology.orchestra.run.smithi043.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00857544 s, 1.2 GB/s 2024-03-21T12:47:55.059 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-03-21T12:47:55.144 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:54 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:47:54.881+0000 7f9555e9a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T12:47:55.144 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:55 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:47:55.143+0000 7f9555e9a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T12:47:55.433 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:55 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:47:55.253+0000 7f9555e9a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T12:47:55.433 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:55 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:47:55.337+0000 7f9555e9a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T12:47:55.433 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:55 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:47:55.432+0000 7f9555e9a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T12:47:55.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:55 smithi043 ceph-mon[30062]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:55.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:55 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:55.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:55 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:55.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:55 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:55.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:55 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:47:55.628 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:55 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:55.629 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:55 smithi043 ceph-mon[30062]: Reconfiguring mgr.y (unknown last config time)... 2024-03-21T12:47:55.629 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:55 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T12:47:55.629 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:55 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T12:47:55.629 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:55 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:55.629 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:55 smithi043 ceph-mon[30062]: Reconfiguring daemon mgr.y on smithi043 2024-03-21T12:47:55.629 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:55 smithi043 ceph-mon[41699]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:55.629 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:55 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:55.629 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:55 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:55.629 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:55 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:55.629 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:55 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:47:55.629 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:55 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:55.629 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:55 smithi043 ceph-mon[41699]: Reconfiguring mgr.y (unknown last config time)... 2024-03-21T12:47:55.629 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:55 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T12:47:55.629 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:55 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T12:47:55.630 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:55 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:55.630 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:55 smithi043 ceph-mon[41699]: Reconfiguring daemon mgr.y on smithi043 2024-03-21T12:47:55.707 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:55 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:47:55.609+0000 7f9555e9a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T12:47:55.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:55 smithi155 ceph-mon[33846]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:55.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:55 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:55.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:55 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:55.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:55 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:55.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:55 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:47:55.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:55 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:55.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:55 smithi155 ceph-mon[33846]: Reconfiguring mgr.y (unknown last config time)... 2024-03-21T12:47:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:55 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T12:47:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:55 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T12:47:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:55 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:55 smithi155 ceph-mon[33846]: Reconfiguring daemon mgr.y on smithi043 2024-03-21T12:47:56.340 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi043:vg_nvme/lv_4 2024-03-21T12:47:56.824 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:47:56.957 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:56 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:47:56.584+0000 7f9555e9a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T12:47:56.957 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:56 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:47:56.713+0000 7f9555e9a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T12:47:57.332 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:57 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:47:57.050+0000 7f9555e9a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T12:47:57.332 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:57 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:47:57.139+0000 7f9555e9a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T12:47:57.332 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:57 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:47:57.331+0000 7f9555e9a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T12:47:57.707 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:57 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:47:57.515+0000 7f9555e9a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T12:47:57.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:57 smithi043 ceph-mon[41699]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:57.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:57 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:57.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:57 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:57.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:57 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:47:57.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:57 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:57.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:57 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:57.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:57 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:47:57.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:57 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:57 smithi043 ceph-mon[30062]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:57 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:57 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:57.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:57 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:47:57.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:57 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:57.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:57 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:57.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:57 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:47:57.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:57 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:58.064 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:57 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:47:57.802+0000 7f9555e9a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T12:47:58.064 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:58 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:47:58.063+0000 7f9555e9a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T12:47:58.064 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:57 smithi155 ceph-mon[33846]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:58.065 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:57 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:58.065 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:57 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:58.065 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:57 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:47:58.065 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:57 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:58.065 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:57 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:58.065 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:57 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:47:58.065 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:57 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:47:58.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.317+0000 7fc46d828700 1 -- 172.21.15.43:0/2612733258 >> v1:172.21.15.43:6789/0 conn(0x7fc4681052a0 legacy=0x7fc468107690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:58.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.317+0000 7fc46d828700 1 -- 172.21.15.43:0/2612733258 shutdown_connections 2024-03-21T12:47:58.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.317+0000 7fc46d828700 1 -- 172.21.15.43:0/2612733258 >> 172.21.15.43:0/2612733258 conn(0x7fc4680fc420 msgr2=0x7fc4680fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:47:58.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.317+0000 7fc46d828700 1 -- 172.21.15.43:0/2612733258 shutdown_connections 2024-03-21T12:47:58.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.317+0000 7fc46d828700 1 -- 172.21.15.43:0/2612733258 wait complete. 2024-03-21T12:47:58.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.317+0000 7fc46d828700 1 Processor -- start 2024-03-21T12:47:58.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.318+0000 7fc46d828700 1 -- start start 2024-03-21T12:47:58.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.318+0000 7fc46d828700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc468197e40 con 0x7fc4681052a0 2024-03-21T12:47:58.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.318+0000 7fc46d828700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc4681a3630 con 0x7fc468108f30 2024-03-21T12:47:58.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.318+0000 7fc46d828700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc4681a4810 con 0x7fc468100e80 2024-03-21T12:47:58.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.318+0000 7fc466ffd700 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7fc468100e80 0x7fc4681972f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:45784/0 (socket says 172.21.15.43:45784) 2024-03-21T12:47:58.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.318+0000 7fc4667fc700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc4681052a0 0x7fc46819d7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58856/0 (socket says 172.21.15.43:58856) 2024-03-21T12:47:58.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.318+0000 7fc466ffd700 1 -- 172.21.15.43:0/1379316646 learned_addr learned my addr 172.21.15.43:0/1379316646 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:47:58.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.318+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2955896961 0 0) 0x7fc4681a4810 con 0x7fc468100e80 2024-03-21T12:47:58.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.318+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4540035b0 con 0x7fc468100e80 2024-03-21T12:47:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.318+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3836805376 0 0) 0x7fc468197e40 con 0x7fc4681052a0 2024-03-21T12:47:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.318+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4681a4810 con 0x7fc4681052a0 2024-03-21T12:47:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.318+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 48904133 0 0) 0x7fc4681a3630 con 0x7fc468108f30 2024-03-21T12:47:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.318+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc468197e40 con 0x7fc468108f30 2024-03-21T12:47:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.319+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1124050918 0 0) 0x7fc4540035b0 con 0x7fc468100e80 2024-03-21T12:47:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.319+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc4681a3630 con 0x7fc468100e80 2024-03-21T12:47:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.319+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fc458001d90 con 0x7fc468100e80 2024-03-21T12:47:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.319+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3607192378 0 0) 0x7fc4681a4810 con 0x7fc4681052a0 2024-03-21T12:47:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.319+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc4540035b0 con 0x7fc4681052a0 2024-03-21T12:47:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.319+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fc4500044f0 con 0x7fc4681052a0 2024-03-21T12:47:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.319+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2132538050 0 0) 0x7fc468197e40 con 0x7fc468108f30 2024-03-21T12:47:58.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.319+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc4681a4810 con 0x7fc468108f30 2024-03-21T12:47:58.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.319+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fc460001830 con 0x7fc468108f30 2024-03-21T12:47:58.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.319+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 <== mon.2 v1:172.21.15.43:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 457736363 0 0) 0x7fc4681a3630 con 0x7fc468100e80 2024-03-21T12:47:58.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.319+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 >> v1:172.21.15.155:6789/0 conn(0x7fc468108f30 legacy=0x7fc4681a0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:58.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.319+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 >> v1:172.21.15.43:6789/0 conn(0x7fc4681052a0 legacy=0x7fc46819d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:47:58.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.319+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 --> v1:172.21.15.43:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4681a5a30 con 0x7fc468100e80 2024-03-21T12:47:58.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.319+0000 7fc46d828700 1 -- 172.21.15.43:0/1379316646 --> v1:172.21.15.43:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4681a2680 con 0x7fc468100e80 2024-03-21T12:47:58.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.319+0000 7fc46d828700 1 -- 172.21.15.43:0/1379316646 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4681a2bc0 con 0x7fc468100e80 2024-03-21T12:47:58.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.319+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 <== mon.2 v1:172.21.15.43:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7fc458002080 con 0x7fc468100e80 2024-03-21T12:47:58.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.319+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 <== mon.2 v1:172.21.15.43:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fc458006130 con 0x7fc468100e80 2024-03-21T12:47:58.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.320+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 <== mon.2 v1:172.21.15.43:6790/0 7 ==== mgrmap(e 13) v1 ==== 46315+0+0 (unknown 152183610 0 0) 0x7fc458011890 con 0x7fc468100e80 2024-03-21T12:47:58.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.320+0000 7fc46d828700 1 -- 172.21.15.43:0/1379316646 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc448005150 con 0x7fc468100e80 2024-03-21T12:47:58.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.320+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 <== mon.2 v1:172.21.15.43:6790/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (unknown 709340102 0 0) 0x7fc458047bc0 con 0x7fc468100e80 2024-03-21T12:47:58.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.325+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 <== mon.2 v1:172.21.15.43:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7fc4580176c0 con 0x7fc468100e80 2024-03-21T12:47:58.457 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:58 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:47:58.298+0000 7f9555e9a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T12:47:58.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:47:58.539+0000 7fc46d828700 1 -- 172.21.15.43:0/1379316646 --> v1:172.21.15.43:6800/4081968544 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc448000bc0 con 0x7fc45403a4b0 2024-03-21T12:47:59.110 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:58 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T12:47:59.110 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:58 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T12:47:59.110 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:58 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:59.156 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:58 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T12:47:59.156 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:58 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T12:47:59.156 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:58 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:59.156 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:58 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T12:47:59.156 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:58 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T12:47:59.156 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:58 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:47:59.457 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:59 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:47:59.109+0000 7f9555e9a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T12:47:59.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:59 smithi155 ceph-mon[33846]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:47:59.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:47:59 smithi155 ceph-mon[33846]: from='client.34115 v1:172.21.15.43:0/1379316646' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:47:59.957 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:59 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:47:59.680+0000 7f9555e9a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T12:47:59.957 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:59 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:47:59.777+0000 7f9555e9a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T12:48:00.231 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:59 smithi043 ceph-mon[30062]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:00.231 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:47:59 smithi043 ceph-mon[30062]: from='client.34115 v1:172.21.15.43:0/1379316646' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:48:00.231 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:59 smithi043 ceph-mon[41699]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:00.231 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:47:59 smithi043 ceph-mon[41699]: from='client.34115 v1:172.21.15.43:0/1379316646' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:48:00.457 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:47:59 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:47:59.993+0000 7f9555e9a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T12:48:00.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:00.816+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 <== mon.2 v1:172.21.15.43:6790/0 10 ==== mgrmap(e 14) v1 ==== 92039+0+0 (unknown 3498086941 0 0) 0x7fc458046460 con 0x7fc468100e80 2024-03-21T12:48:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:00 smithi155 ceph-mon[33846]: Standby manager daemon x started 2024-03-21T12:48:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:00 smithi155 ceph-mon[33846]: from='mgr.? v1:172.21.15.155:0/153244718' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-21T12:48:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:00 smithi155 ceph-mon[33846]: from='mgr.? v1:172.21.15.155:0/153244718' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T12:48:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:00 smithi155 ceph-mon[33846]: from='mgr.? v1:172.21.15.155:0/153244718' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-21T12:48:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:00 smithi155 ceph-mon[33846]: from='mgr.? v1:172.21.15.155:0/153244718' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T12:48:01.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:00 smithi043 ceph-mon[30062]: Standby manager daemon x started 2024-03-21T12:48:01.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:00 smithi043 ceph-mon[30062]: from='mgr.? v1:172.21.15.155:0/153244718' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-21T12:48:01.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:00 smithi043 ceph-mon[30062]: from='mgr.? v1:172.21.15.155:0/153244718' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T12:48:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:00 smithi043 ceph-mon[30062]: from='mgr.? v1:172.21.15.155:0/153244718' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-21T12:48:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:00 smithi043 ceph-mon[30062]: from='mgr.? v1:172.21.15.155:0/153244718' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T12:48:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:00 smithi043 ceph-mon[41699]: Standby manager daemon x started 2024-03-21T12:48:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:00 smithi043 ceph-mon[41699]: from='mgr.? v1:172.21.15.155:0/153244718' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-21T12:48:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:00 smithi043 ceph-mon[41699]: from='mgr.? v1:172.21.15.155:0/153244718' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T12:48:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:00 smithi043 ceph-mon[41699]: from='mgr.? v1:172.21.15.155:0/153244718' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-21T12:48:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:00 smithi043 ceph-mon[41699]: from='mgr.? v1:172.21.15.155:0/153244718' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T12:48:02.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:01 smithi155 ceph-mon[33846]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:02.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:01 smithi155 ceph-mon[33846]: mgrmap e14: y(active, since 68s), standbys: x 2024-03-21T12:48:02.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:01 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T12:48:02.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:01 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3633345598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c030e54-4f13-460e-8ec3-940135cee693"}]: dispatch 2024-03-21T12:48:02.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:01 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3633345598' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c030e54-4f13-460e-8ec3-940135cee693"}]': finished 2024-03-21T12:48:02.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:01 smithi155 ceph-mon[33846]: osdmap e5: 1 total, 0 up, 1 in 2024-03-21T12:48:02.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:01 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T12:48:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:01 smithi043 ceph-mon[41699]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:01 smithi043 ceph-mon[41699]: mgrmap e14: y(active, since 68s), standbys: x 2024-03-21T12:48:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:01 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T12:48:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:01 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3633345598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c030e54-4f13-460e-8ec3-940135cee693"}]: dispatch 2024-03-21T12:48:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:01 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3633345598' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c030e54-4f13-460e-8ec3-940135cee693"}]': finished 2024-03-21T12:48:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:01 smithi043 ceph-mon[41699]: osdmap e5: 1 total, 0 up, 1 in 2024-03-21T12:48:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:01 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T12:48:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:01 smithi043 ceph-mon[30062]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:01 smithi043 ceph-mon[30062]: mgrmap e14: y(active, since 68s), standbys: x 2024-03-21T12:48:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:01 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T12:48:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:01 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3633345598' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3c030e54-4f13-460e-8ec3-940135cee693"}]: dispatch 2024-03-21T12:48:02.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:01 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3633345598' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3c030e54-4f13-460e-8ec3-940135cee693"}]': finished 2024-03-21T12:48:02.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:01 smithi043 ceph-mon[30062]: osdmap e5: 1 total, 0 up, 1 in 2024-03-21T12:48:02.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:01 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T12:48:03.125 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:02 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2682022909' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T12:48:03.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:02 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2682022909' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T12:48:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:02 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2682022909' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T12:48:04.095 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:03 smithi043 ceph-mon[30062]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:04.095 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:03 smithi043 ceph-mon[41699]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:03 smithi155 ceph-mon[33846]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:06.090 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:05 smithi043 ceph-mon[30062]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:06.090 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:05 smithi043 ceph-mon[41699]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:05 smithi155 ceph-mon[33846]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:07.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:07 smithi043 ceph-mon[30062]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:07.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:07 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T12:48:07.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:07 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:48:07.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:07 smithi043 ceph-mon[41699]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:07.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:07 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T12:48:07.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:07 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:48:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:07 smithi155 ceph-mon[33846]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:07 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T12:48:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:07 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:48:08.977 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:08 smithi043 ceph-mon[30062]: Deploying daemon osd.0 on smithi043 2024-03-21T12:48:08.977 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:08 smithi043 ceph-mon[41699]: Deploying daemon osd.0 on smithi043 2024-03-21T12:48:09.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:08 smithi155 ceph-mon[33846]: Deploying daemon osd.0 on smithi043 2024-03-21T12:48:10.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:09 smithi155 ceph-mon[33846]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:10.226 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:09 smithi043 ceph-mon[30062]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:10.226 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:09 smithi043 ceph-mon[41699]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:11 smithi155 ceph-mon[33846]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:12.226 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:11 smithi043 ceph-mon[30062]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:12.226 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:11 smithi043 ceph-mon[41699]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:14.050 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:13 smithi043 ceph-mon[30062]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:14.050 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:13 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:14.050 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:13 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:14.050 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:13 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:48:14.050 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:13 smithi043 ceph-mon[41699]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:14.050 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:13 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:14.050 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:13 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:14.051 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:13 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:48:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:13 smithi155 ceph-mon[33846]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:13 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:13 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:13 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:48:16.169 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:15 smithi043 ceph-mon[30062]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:16.169 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:15 smithi043 ceph-mon[41699]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:16.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:15 smithi155 ceph-mon[33846]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:16.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:16.791+0000 7fc46c826700 1 -- 172.21.15.43:0/1379316646 <== mgr.14152 v1:172.21.15.43:6800/4081968544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 2375882481) 0x7fc448000bc0 con 0x7fc45403a4b0 2024-03-21T12:48:16.792 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 0 on host 'smithi043' 2024-03-21T12:48:16.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:16.793+0000 7fc46d828700 1 -- 172.21.15.43:0/1379316646 >> v1:172.21.15.43:6800/4081968544 conn(0x7fc45403a4b0 legacy=0x7fc45403c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:48:16.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:16.793+0000 7fc46d828700 1 -- 172.21.15.43:0/1379316646 >> v1:172.21.15.43:6790/0 conn(0x7fc468100e80 legacy=0x7fc4681972f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:48:16.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:16.793+0000 7fc46d828700 1 -- 172.21.15.43:0/1379316646 shutdown_connections 2024-03-21T12:48:16.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:16.793+0000 7fc46d828700 1 -- 172.21.15.43:0/1379316646 >> 172.21.15.43:0/1379316646 conn(0x7fc4680fc420 msgr2=0x7fc4680fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:48:16.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:16.794+0000 7fc46d828700 1 -- 172.21.15.43:0/1379316646 shutdown_connections 2024-03-21T12:48:16.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:16.794+0000 7fc46d828700 1 -- 172.21.15.43:0/1379316646 wait complete. 2024-03-21T12:48:17.411 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:17 smithi043 ceph-mon[30062]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:17.411 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:17 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:17.412 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:17 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:17.412 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:17 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:48:17.412 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:17 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:48:17.412 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:17 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:17.412 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:17 smithi043 ceph-mon[30062]: from='osd.0 v1:172.21.15.43:6801/3260944688' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T12:48:17.412 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:17 smithi043 ceph-mon[30062]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T12:48:17.412 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:17 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:17.412 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:17 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:17.412 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:17 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:48:17.412 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:17 smithi043 ceph-mon[41699]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:17.412 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:17 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:17.412 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:17 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:17.413 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:17 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:48:17.413 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:17 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:48:17.413 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:17 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:17.413 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:17 smithi043 ceph-mon[41699]: from='osd.0 v1:172.21.15.43:6801/3260944688' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T12:48:17.413 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:17 smithi043 ceph-mon[41699]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T12:48:17.413 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:17 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:17.413 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:17 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:17.413 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:17 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:48:17.437 DEBUG:teuthology.orchestra.run.smithi043:osd.0> sudo journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@osd.0.service 2024-03-21T12:48:17.440 INFO:tasks.cephadm:Deploying osd.1 on smithi043 with /dev/vg_nvme/lv_3... 2024-03-21T12:48:17.440 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-21T12:48:17.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:17 smithi155 ceph-mon[33846]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:17.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:17 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:17.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:17 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:17.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:17 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:48:17.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:17 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:48:17.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:17 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:17.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:17 smithi155 ceph-mon[33846]: from='osd.0 v1:172.21.15.43:6801/3260944688' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T12:48:17.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:17 smithi155 ceph-mon[33846]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T12:48:17.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:17 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:17.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:17 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:17.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:17 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:48:18.310 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:18 smithi043 ceph-mon[30062]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-21T12:48:18.310 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:18 smithi043 ceph-mon[30062]: from='osd.0 v1:172.21.15.43:6801/3260944688' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-21T12:48:18.310 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:18 smithi043 ceph-mon[30062]: osdmap e6: 1 total, 0 up, 1 in 2024-03-21T12:48:18.310 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:18 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T12:48:18.310 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:18 smithi043 ceph-mon[30062]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-21T12:48:18.311 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:18 smithi043 ceph-mon[41699]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-21T12:48:18.311 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:18 smithi043 ceph-mon[41699]: from='osd.0 v1:172.21.15.43:6801/3260944688' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-21T12:48:18.311 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:18 smithi043 ceph-mon[41699]: osdmap e6: 1 total, 0 up, 1 in 2024-03-21T12:48:18.311 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:18 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T12:48:18.311 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:18 smithi043 ceph-mon[41699]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-21T12:48:18.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:18 smithi155 ceph-mon[33846]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-21T12:48:18.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:18 smithi155 ceph-mon[33846]: from='osd.0 v1:172.21.15.43:6801/3260944688' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-21T12:48:18.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:18 smithi155 ceph-mon[33846]: osdmap e6: 1 total, 0 up, 1 in 2024-03-21T12:48:18.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:18 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T12:48:18.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:18 smithi155 ceph-mon[33846]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-21T12:48:18.594 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 21 12:48:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-0[46126]: 2024-03-21T12:48:18.316+0000 7f615e801700 -1 osd.0 0 waiting for initial osdmap 2024-03-21T12:48:18.594 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 21 12:48:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-0[46126]: 2024-03-21T12:48:18.318+0000 7f61565ec700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T12:48:18.598 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:48:19.208 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:19 smithi043 ceph-mon[30062]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:19.208 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:19 smithi043 ceph-mon[30062]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-03-21T12:48:19.208 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:19 smithi043 ceph-mon[30062]: osdmap e7: 1 total, 0 up, 1 in 2024-03-21T12:48:19.208 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:19 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T12:48:19.208 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:19 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T12:48:19.208 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:19 smithi043 ceph-mon[30062]: from='osd.0 ' entity='osd.0' 2024-03-21T12:48:19.209 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:19 smithi043 ceph-mon[41699]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:19.209 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:19 smithi043 ceph-mon[41699]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-03-21T12:48:19.209 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:19 smithi043 ceph-mon[41699]: osdmap e7: 1 total, 0 up, 1 in 2024-03-21T12:48:19.209 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:19 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T12:48:19.209 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:19 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T12:48:19.209 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:19 smithi043 ceph-mon[41699]: from='osd.0 ' entity='osd.0' 2024-03-21T12:48:19.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:19 smithi155 ceph-mon[33846]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:19.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:19 smithi155 ceph-mon[33846]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-03-21T12:48:19.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:19 smithi155 ceph-mon[33846]: osdmap e7: 1 total, 0 up, 1 in 2024-03-21T12:48:19.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:19 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T12:48:19.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:19 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T12:48:19.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:19 smithi155 ceph-mon[33846]: from='osd.0 ' entity='osd.0' 2024-03-21T12:48:20.365 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:20 smithi043 ceph-mon[30062]: purged_snaps scrub starts 2024-03-21T12:48:20.365 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:20 smithi043 ceph-mon[30062]: purged_snaps scrub ok 2024-03-21T12:48:20.365 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:20 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T12:48:20.365 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:20 smithi043 ceph-mon[30062]: osd.0 v1:172.21.15.43:6801/3260944688 boot 2024-03-21T12:48:20.365 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:20 smithi043 ceph-mon[30062]: osdmap e8: 1 total, 1 up, 1 in 2024-03-21T12:48:20.365 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:20 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T12:48:20.365 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:20 smithi043 ceph-mon[41699]: purged_snaps scrub starts 2024-03-21T12:48:20.366 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:20 smithi043 ceph-mon[41699]: purged_snaps scrub ok 2024-03-21T12:48:20.366 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:20 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T12:48:20.366 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:20 smithi043 ceph-mon[41699]: osd.0 v1:172.21.15.43:6801/3260944688 boot 2024-03-21T12:48:20.366 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:20 smithi043 ceph-mon[41699]: osdmap e8: 1 total, 1 up, 1 in 2024-03-21T12:48:20.366 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:20 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T12:48:20.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:20 smithi155 ceph-mon[33846]: purged_snaps scrub starts 2024-03-21T12:48:20.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:20 smithi155 ceph-mon[33846]: purged_snaps scrub ok 2024-03-21T12:48:20.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:20 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T12:48:20.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:20 smithi155 ceph-mon[33846]: osd.0 v1:172.21.15.43:6801/3260944688 boot 2024-03-21T12:48:20.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:20 smithi155 ceph-mon[33846]: osdmap e8: 1 total, 1 up, 1 in 2024-03-21T12:48:20.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:20 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T12:48:21.576 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:21 smithi043 ceph-mon[30062]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:21.577 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:21 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:21.577 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:21 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:21.577 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:21 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:48:21.577 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:21 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:21.577 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:21 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:48:21.577 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:21 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:48:21.577 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:21 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:21.577 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:21 smithi043 ceph-mon[30062]: osdmap e9: 1 total, 1 up, 1 in 2024-03-21T12:48:21.577 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-21T12:48:21.577 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-21T12:48:21.578 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-03-21T12:48:21.578 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-03-21T12:48:21.578 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00868498 s, 1.2 GB/s 2024-03-21T12:48:21.578 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-03-21T12:48:21.578 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:21 smithi043 ceph-mon[41699]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:21.578 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:21 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:21.578 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:21 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:21.578 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:21 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:48:21.578 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:21 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:21.578 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:21 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:48:21.578 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:21 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:48:21.579 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:21 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:21.579 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:21 smithi043 ceph-mon[41699]: osdmap e9: 1 total, 1 up, 1 in 2024-03-21T12:48:21.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:21 smithi155 ceph-mon[33846]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T12:48:21.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:21 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:21.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:21 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:21.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:21 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:48:21.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:21 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:21.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:21 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:48:21.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:21 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:48:21.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:21 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:21.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:21 smithi155 ceph-mon[33846]: osdmap e9: 1 total, 1 up, 1 in 2024-03-21T12:48:22.226 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi043:vg_nvme/lv_3 2024-03-21T12:48:22.469 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:48:22.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:22 smithi043 ceph-mon[30062]: Detected new or changed devices on smithi043 2024-03-21T12:48:22.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:22 smithi043 ceph-mon[30062]: Adjusting osd_memory_target on smithi043 to 16069M 2024-03-21T12:48:22.483 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:22 smithi043 ceph-mon[41699]: Detected new or changed devices on smithi043 2024-03-21T12:48:22.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:22 smithi043 ceph-mon[41699]: Adjusting osd_memory_target on smithi043 to 16069M 2024-03-21T12:48:22.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:22 smithi155 ceph-mon[33846]: Detected new or changed devices on smithi043 2024-03-21T12:48:22.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:22 smithi155 ceph-mon[33846]: Adjusting osd_memory_target on smithi043 to 16069M 2024-03-21T12:48:23.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:23 smithi043 ceph-mon[30062]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:23.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:23 smithi043 ceph-mon[41699]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:23.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:23 smithi155 ceph-mon[33846]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:24.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.142+0000 7f45e6ab9700 1 -- 172.21.15.43:0/4020222271 >> v1:172.21.15.43:6789/0 conn(0x7f45e0101fa0 legacy=0x7f45e01023c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:48:24.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.142+0000 7f45e6ab9700 1 -- 172.21.15.43:0/4020222271 shutdown_connections 2024-03-21T12:48:24.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.142+0000 7f45e6ab9700 1 -- 172.21.15.43:0/4020222271 >> 172.21.15.43:0/4020222271 conn(0x7f45e00fd540 msgr2=0x7f45e00ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:48:24.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.142+0000 7f45e6ab9700 1 -- 172.21.15.43:0/4020222271 shutdown_connections 2024-03-21T12:48:24.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.142+0000 7f45e6ab9700 1 -- 172.21.15.43:0/4020222271 wait complete. 2024-03-21T12:48:24.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.143+0000 7f45e6ab9700 1 Processor -- start 2024-03-21T12:48:24.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.143+0000 7f45e6ab9700 1 -- start start 2024-03-21T12:48:24.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.143+0000 7f45e6ab9700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f45e01a45c0 con 0x7f45e0101fa0 2024-03-21T12:48:24.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.143+0000 7f45e6ab9700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f45e01a5800 con 0x7f45e01063d0 2024-03-21T12:48:24.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.143+0000 7f45e6ab9700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f45e01a6a40 con 0x7f45e010a110 2024-03-21T12:48:24.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.143+0000 7f45e4855700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f45e0101fa0 0x7f45e0075e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:50132/0 (socket says 172.21.15.43:50132) 2024-03-21T12:48:24.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.143+0000 7f45e4855700 1 -- 172.21.15.43:0/2282656250 learned_addr learned my addr 172.21.15.43:0/2282656250 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:48:24.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.144+0000 7f45ddffb700 1 -- 172.21.15.43:0/2282656250 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112930114 0 0) 0x7f45e01a6a40 con 0x7f45e010a110 2024-03-21T12:48:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.144+0000 7f45ddffb700 1 -- 172.21.15.43:0/2282656250 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45d00031f0 con 0x7f45e010a110 2024-03-21T12:48:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.144+0000 7f45ddffb700 1 -- 172.21.15.43:0/2282656250 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2323283953 0 0) 0x7f45e01a45c0 con 0x7f45e0101fa0 2024-03-21T12:48:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.144+0000 7f45ddffb700 1 -- 172.21.15.43:0/2282656250 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45e01a6a40 con 0x7f45e0101fa0 2024-03-21T12:48:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.144+0000 7f45ddffb700 1 -- 172.21.15.43:0/2282656250 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1070475992 0 0) 0x7f45e01a5800 con 0x7f45e01063d0 2024-03-21T12:48:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.144+0000 7f45ddffb700 1 -- 172.21.15.43:0/2282656250 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45e01a45c0 con 0x7f45e01063d0 2024-03-21T12:48:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.144+0000 7f45ddffb700 1 -- 172.21.15.43:0/2282656250 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3445386310 0 0) 0x7f45d00031f0 con 0x7f45e010a110 2024-03-21T12:48:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.144+0000 7f45ddffb700 1 -- 172.21.15.43:0/2282656250 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f45e01a5800 con 0x7f45e010a110 2024-03-21T12:48:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.144+0000 7f45ddffb700 1 -- 172.21.15.43:0/2282656250 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f45d8001b80 con 0x7f45e010a110 2024-03-21T12:48:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.144+0000 7f45ddffb700 1 -- 172.21.15.43:0/2282656250 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2402759519 0 0) 0x7f45e01a6a40 con 0x7f45e0101fa0 2024-03-21T12:48:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.144+0000 7f45ddffb700 1 -- 172.21.15.43:0/2282656250 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f45d00031f0 con 0x7f45e0101fa0 2024-03-21T12:48:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.144+0000 7f45ddffb700 1 -- 172.21.15.43:0/2282656250 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f45d4004680 con 0x7f45e0101fa0 2024-03-21T12:48:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.144+0000 7f45ddffb700 1 -- 172.21.15.43:0/2282656250 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3155041558 0 0) 0x7f45e01a45c0 con 0x7f45e01063d0 2024-03-21T12:48:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.144+0000 7f45ddffb700 1 -- 172.21.15.43:0/2282656250 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f45e01a6a40 con 0x7f45e01063d0 2024-03-21T12:48:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.144+0000 7f45ddffb700 1 -- 172.21.15.43:0/2282656250 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f45cc0018b0 con 0x7f45e01063d0 2024-03-21T12:48:24.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.144+0000 7f45ddffb700 1 -- 172.21.15.43:0/2282656250 <== mon.2 v1:172.21.15.43:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2976514959 0 0) 0x7f45e01a5800 con 0x7f45e010a110 2024-03-21T12:48:24.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.144+0000 7f45ddffb700 1 -- 172.21.15.43:0/2282656250 >> v1:172.21.15.155:6789/0 conn(0x7f45e01063d0 legacy=0x7f45e019f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:48:24.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.144+0000 7f45ddffb700 1 -- 172.21.15.43:0/2282656250 >> v1:172.21.15.43:6789/0 conn(0x7f45e0101fa0 legacy=0x7f45e0075e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:48:24.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.144+0000 7f45ddffb700 1 -- 172.21.15.43:0/2282656250 --> v1:172.21.15.43:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e01a7c80 con 0x7f45e010a110 2024-03-21T12:48:24.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.144+0000 7f45e6ab9700 1 -- 172.21.15.43:0/2282656250 --> v1:172.21.15.43:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e01a47f0 con 0x7f45e010a110 2024-03-21T12:48:24.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.144+0000 7f45ddffb700 1 -- 172.21.15.43:0/2282656250 <== mon.2 v1:172.21.15.43:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f45d8002090 con 0x7f45e010a110 2024-03-21T12:48:24.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.144+0000 7f45ddffb700 1 -- 172.21.15.43:0/2282656250 <== mon.2 v1:172.21.15.43:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f45d8004c40 con 0x7f45e010a110 2024-03-21T12:48:24.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.145+0000 7f45e6ab9700 1 -- 172.21.15.43:0/2282656250 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e01a4d30 con 0x7f45e010a110 2024-03-21T12:48:24.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.145+0000 7f45e6ab9700 1 -- 172.21.15.43:0/2282656250 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e0110b70 con 0x7f45e010a110 2024-03-21T12:48:24.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.146+0000 7f45ddffb700 1 -- 172.21.15.43:0/2282656250 <== mon.2 v1:172.21.15.43:6790/0 7 ==== mgrmap(e 14) v1 ==== 92039+0+0 (unknown 3498086941 0 0) 0x7f45d8003a20 con 0x7f45e010a110 2024-03-21T12:48:24.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.147+0000 7f45ddffb700 1 -- 172.21.15.43:0/2282656250 <== mon.2 v1:172.21.15.43:6790/0 8 ==== osd_map(9..9 src has 1..9) v4 ==== 1541+0+0 (unknown 3864353022 0 0) 0x7f45d8087380 con 0x7f45e010a110 2024-03-21T12:48:24.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.152+0000 7f45ddffb700 1 -- 172.21.15.43:0/2282656250 <== mon.2 v1:172.21.15.43:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f45d8055d00 con 0x7f45e010a110 2024-03-21T12:48:24.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:24.366+0000 7f45e6ab9700 1 -- 172.21.15.43:0/2282656250 --> v1:172.21.15.43:6800/4081968544 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f45e01a5090 con 0x7f45d006f1d0 2024-03-21T12:48:25.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:25 smithi043 ceph-mon[30062]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:25.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:25 smithi043 ceph-mon[30062]: from='client.34142 v1:172.21.15.43:0/2282656250' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:48:25.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:25 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T12:48:25.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:25 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T12:48:25.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:25 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:48:25.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:25 smithi043 ceph-mon[41699]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:25.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:25 smithi043 ceph-mon[41699]: from='client.34142 v1:172.21.15.43:0/2282656250' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:48:25.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:25 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T12:48:25.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:25 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T12:48:25.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:25 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:48:25.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:25 smithi155 ceph-mon[33846]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:25.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:25 smithi155 ceph-mon[33846]: from='client.34142 v1:172.21.15.43:0/2282656250' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:48:25.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:25 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T12:48:25.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:25 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T12:48:25.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:25 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:48:27.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:27 smithi155 ceph-mon[33846]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:27.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:27 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1472034576' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fcc9c3e0-665d-4331-abf3-3ea6e2ac2c63"}]: dispatch 2024-03-21T12:48:27.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:27 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1472034576' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fcc9c3e0-665d-4331-abf3-3ea6e2ac2c63"}]': finished 2024-03-21T12:48:27.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:27 smithi155 ceph-mon[33846]: osdmap e10: 2 total, 1 up, 2 in 2024-03-21T12:48:27.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:27 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T12:48:27.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:27 smithi043 ceph-mon[41699]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:27.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:27 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1472034576' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fcc9c3e0-665d-4331-abf3-3ea6e2ac2c63"}]: dispatch 2024-03-21T12:48:27.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:27 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1472034576' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fcc9c3e0-665d-4331-abf3-3ea6e2ac2c63"}]': finished 2024-03-21T12:48:27.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:27 smithi043 ceph-mon[41699]: osdmap e10: 2 total, 1 up, 2 in 2024-03-21T12:48:27.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:27 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T12:48:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:27 smithi043 ceph-mon[30062]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:27 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1472034576' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fcc9c3e0-665d-4331-abf3-3ea6e2ac2c63"}]: dispatch 2024-03-21T12:48:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:27 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1472034576' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fcc9c3e0-665d-4331-abf3-3ea6e2ac2c63"}]': finished 2024-03-21T12:48:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:27 smithi043 ceph-mon[30062]: osdmap e10: 2 total, 1 up, 2 in 2024-03-21T12:48:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:27 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T12:48:28.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2182907524' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T12:48:28.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2182907524' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T12:48:28.733 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2182907524' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T12:48:29.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:29 smithi155 ceph-mon[33846]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:29.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:29 smithi043 ceph-mon[30062]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:29.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:29 smithi043 ceph-mon[41699]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:31.651 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:31 smithi043 ceph-mon[30062]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:31.651 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:31 smithi043 ceph-mon[41699]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:31.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:31 smithi155 ceph-mon[33846]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:33.435 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:33 smithi043 ceph-mon[30062]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:33.435 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:33 smithi043 ceph-mon[41699]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:33.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:33 smithi155 ceph-mon[33846]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:34.613 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:34 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T12:48:34.614 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:34 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:48:34.614 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:34 smithi043 ceph-mon[30062]: Deploying daemon osd.1 on smithi043 2024-03-21T12:48:34.614 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:34 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T12:48:34.614 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:34 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:48:34.614 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:34 smithi043 ceph-mon[41699]: Deploying daemon osd.1 on smithi043 2024-03-21T12:48:34.701 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:34 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T12:48:34.701 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:34 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:48:34.701 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:34 smithi155 ceph-mon[33846]: Deploying daemon osd.1 on smithi043 2024-03-21T12:48:35.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:35 smithi043 ceph-mon[30062]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:35.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:35 smithi043 ceph-mon[41699]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:35.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:35 smithi155 ceph-mon[33846]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:37.680 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:37 smithi043 ceph-mon[30062]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:37.680 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:37 smithi043 ceph-mon[41699]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:37.706 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:37 smithi155 ceph-mon[33846]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:39.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:39 smithi155 ceph-mon[33846]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:39.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:39 smithi043 ceph-mon[30062]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:39.733 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:39 smithi043 ceph-mon[41699]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:40.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:40 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:40.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:40 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:40.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:40 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:48:40.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:40 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:40.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:40 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:40.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:40 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:48:40.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:40 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:40.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:40 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:40.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:40 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:48:41.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:41 smithi155 ceph-mon[33846]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:41.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:41 smithi043 ceph-mon[30062]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:41.733 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:41 smithi043 ceph-mon[41699]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:43.173 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 1 on host 'smithi043' 2024-03-21T12:48:43.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:43.172+0000 7f45ddffb700 1 -- 172.21.15.43:0/2282656250 <== mgr.14152 v1:172.21.15.43:6800/4081968544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 3515414597) 0x7f45e01a5090 con 0x7f45d006f1d0 2024-03-21T12:48:43.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:43.175+0000 7f45e6ab9700 1 -- 172.21.15.43:0/2282656250 >> v1:172.21.15.43:6800/4081968544 conn(0x7f45d006f1d0 legacy=0x7f45d0071690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:48:43.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:43.175+0000 7f45e6ab9700 1 -- 172.21.15.43:0/2282656250 >> v1:172.21.15.43:6790/0 conn(0x7f45e010a110 legacy=0x7f45e01a2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:48:43.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:43.175+0000 7f45e6ab9700 1 -- 172.21.15.43:0/2282656250 shutdown_connections 2024-03-21T12:48:43.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:43.175+0000 7f45e6ab9700 1 -- 172.21.15.43:0/2282656250 >> 172.21.15.43:0/2282656250 conn(0x7f45e00fd540 msgr2=0x7f45e0108820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:48:43.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:43.175+0000 7f45e6ab9700 1 -- 172.21.15.43:0/2282656250 shutdown_connections 2024-03-21T12:48:43.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:43.175+0000 7f45e6ab9700 1 -- 172.21.15.43:0/2282656250 wait complete. 2024-03-21T12:48:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:43 smithi155 ceph-mon[33846]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:43 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:43 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:43 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:48:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:43 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:48:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:43 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:43 smithi155 ceph-mon[33846]: from='osd.1 v1:172.21.15.43:6805/1862805780' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T12:48:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:43 smithi155 ceph-mon[33846]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T12:48:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:43 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:43 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:43 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:48:43.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:43 smithi043 ceph-mon[30062]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:43 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:43 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:43 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:48:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:43 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:48:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:43 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:43 smithi043 ceph-mon[30062]: from='osd.1 v1:172.21.15.43:6805/1862805780' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T12:48:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:43 smithi043 ceph-mon[30062]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T12:48:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:43 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:43 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:43 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:48:43.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:43 smithi043 ceph-mon[41699]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:43.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:43 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:43 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:43 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:48:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:43 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:48:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:43 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:43 smithi043 ceph-mon[41699]: from='osd.1 v1:172.21.15.43:6805/1862805780' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T12:48:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:43 smithi043 ceph-mon[41699]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T12:48:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:43 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:43 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:43 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:48:43.808 DEBUG:teuthology.orchestra.run.smithi043:osd.1> sudo journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@osd.1.service 2024-03-21T12:48:43.811 INFO:tasks.cephadm:Deploying osd.2 on smithi043 with /dev/vg_nvme/lv_2... 2024-03-21T12:48:43.811 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-21T12:48:44.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:44 smithi155 ceph-mon[33846]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-21T12:48:44.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:44 smithi155 ceph-mon[33846]: osdmap e11: 2 total, 1 up, 2 in 2024-03-21T12:48:44.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:44 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T12:48:44.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:44 smithi155 ceph-mon[33846]: from='osd.1 v1:172.21.15.43:6805/1862805780' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-21T12:48:44.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:44 smithi155 ceph-mon[33846]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-21T12:48:44.595 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:48:44.712 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:44 smithi043 ceph-mon[41699]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-21T12:48:44.712 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:44 smithi043 ceph-mon[41699]: osdmap e11: 2 total, 1 up, 2 in 2024-03-21T12:48:44.712 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:44 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T12:48:44.713 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:44 smithi043 ceph-mon[41699]: from='osd.1 v1:172.21.15.43:6805/1862805780' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-21T12:48:44.713 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:44 smithi043 ceph-mon[41699]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-21T12:48:44.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:44 smithi043 ceph-mon[30062]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-21T12:48:44.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:44 smithi043 ceph-mon[30062]: osdmap e11: 2 total, 1 up, 2 in 2024-03-21T12:48:44.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:44 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T12:48:44.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:44 smithi043 ceph-mon[30062]: from='osd.1 v1:172.21.15.43:6805/1862805780' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-21T12:48:44.713 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:44 smithi043 ceph-mon[30062]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-21T12:48:44.713 INFO:journalctl@ceph.osd.1.smithi043.stdout:Mar 21 12:48:44 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-1[52242]: 2024-03-21T12:48:44.443+0000 7fd9f07f9700 -1 osd.1 0 waiting for initial osdmap 2024-03-21T12:48:44.714 INFO:journalctl@ceph.osd.1.smithi043.stdout:Mar 21 12:48:44 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-1[52242]: 2024-03-21T12:48:44.446+0000 7fd9ec5ec700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T12:48:45.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:45 smithi155 ceph-mon[33846]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:45.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:45 smithi155 ceph-mon[33846]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-03-21T12:48:45.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:45 smithi155 ceph-mon[33846]: osdmap e12: 2 total, 1 up, 2 in 2024-03-21T12:48:45.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:45 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T12:48:45.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:45 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T12:48:45.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:45 smithi043 ceph-mon[30062]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:45.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:45 smithi043 ceph-mon[30062]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-03-21T12:48:45.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:45 smithi043 ceph-mon[30062]: osdmap e12: 2 total, 1 up, 2 in 2024-03-21T12:48:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:45 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T12:48:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:45 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T12:48:45.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:45 smithi043 ceph-mon[41699]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:45.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:45 smithi043 ceph-mon[41699]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-03-21T12:48:45.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:45 smithi043 ceph-mon[41699]: osdmap e12: 2 total, 1 up, 2 in 2024-03-21T12:48:45.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:45 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T12:48:45.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:45 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T12:48:46.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:46 smithi043 ceph-mon[30062]: purged_snaps scrub starts 2024-03-21T12:48:46.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:46 smithi043 ceph-mon[30062]: purged_snaps scrub ok 2024-03-21T12:48:46.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:46 smithi043 ceph-mon[30062]: osd.1 v1:172.21.15.43:6805/1862805780 boot 2024-03-21T12:48:46.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:46 smithi043 ceph-mon[30062]: osdmap e13: 2 total, 2 up, 2 in 2024-03-21T12:48:46.635 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:46 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T12:48:46.635 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:46 smithi043 ceph-mon[41699]: purged_snaps scrub starts 2024-03-21T12:48:46.635 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:46 smithi043 ceph-mon[41699]: purged_snaps scrub ok 2024-03-21T12:48:46.635 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:46 smithi043 ceph-mon[41699]: osd.1 v1:172.21.15.43:6805/1862805780 boot 2024-03-21T12:48:46.635 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:46 smithi043 ceph-mon[41699]: osdmap e13: 2 total, 2 up, 2 in 2024-03-21T12:48:46.635 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:46 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T12:48:46.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:46 smithi155 ceph-mon[33846]: purged_snaps scrub starts 2024-03-21T12:48:46.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:46 smithi155 ceph-mon[33846]: purged_snaps scrub ok 2024-03-21T12:48:46.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:46 smithi155 ceph-mon[33846]: osd.1 v1:172.21.15.43:6805/1862805780 boot 2024-03-21T12:48:46.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:46 smithi155 ceph-mon[33846]: osdmap e13: 2 total, 2 up, 2 in 2024-03-21T12:48:46.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:46 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T12:48:47.198 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-21T12:48:47.199 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-21T12:48:47.199 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-03-21T12:48:47.199 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-03-21T12:48:47.199 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00972748 s, 1.1 GB/s 2024-03-21T12:48:47.199 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-03-21T12:48:47.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:47 smithi043 ceph-mon[30062]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:47.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:47 smithi043 ceph-mon[30062]: osdmap e14: 2 total, 2 up, 2 in 2024-03-21T12:48:47.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:47 smithi043 ceph-mon[41699]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:47.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:47 smithi043 ceph-mon[41699]: osdmap e14: 2 total, 2 up, 2 in 2024-03-21T12:48:47.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:47 smithi155 ceph-mon[33846]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-03-21T12:48:47.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:47 smithi155 ceph-mon[33846]: osdmap e14: 2 total, 2 up, 2 in 2024-03-21T12:48:48.060 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi043:vg_nvme/lv_2 2024-03-21T12:48:48.639 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:48:49.707 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:49 smithi043 ceph-mon[30062]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:48:49.707 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:49 smithi043 ceph-mon[30062]: Detected new or changed devices on smithi043 2024-03-21T12:48:49.707 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:49 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:49.708 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:49 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:49.708 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:49 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:48:49.708 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:49 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:48:49.708 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:49 smithi043 ceph-mon[30062]: Adjusting osd_memory_target on smithi043 to 8034M 2024-03-21T12:48:49.708 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:49 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:49.708 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:49 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:48:49.708 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:49 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:48:49.708 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:49 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:49.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:49 smithi043 ceph-mon[41699]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:48:49.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:49 smithi043 ceph-mon[41699]: Detected new or changed devices on smithi043 2024-03-21T12:48:49.708 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:49 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:49.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:49 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:49.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:49 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:48:49.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:49 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:48:49.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:49 smithi043 ceph-mon[41699]: Adjusting osd_memory_target on smithi043 to 8034M 2024-03-21T12:48:49.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:49 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:49.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:49 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:48:49.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:49 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:48:49.709 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:49 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:49.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:49 smithi155 ceph-mon[33846]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:48:49.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:49 smithi155 ceph-mon[33846]: Detected new or changed devices on smithi043 2024-03-21T12:48:49.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:49 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:49.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:49 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:49.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:49 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:48:49.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:49 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:48:49.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:49 smithi155 ceph-mon[33846]: Adjusting osd_memory_target on smithi043 to 8034M 2024-03-21T12:48:49.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:49 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:49.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:49 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:48:49.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:49 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:48:49.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:49 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:48:50.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.242+0000 7f8b0094b700 1 -- 172.21.15.43:0/2499553470 >> v1:172.21.15.43:6790/0 conn(0x7f8afc1087a0 legacy=0x7f8afc10ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:48:50.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.242+0000 7f8b0094b700 1 -- 172.21.15.43:0/2499553470 shutdown_connections 2024-03-21T12:48:50.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.242+0000 7f8b0094b700 1 -- 172.21.15.43:0/2499553470 >> 172.21.15.43:0/2499553470 conn(0x7f8afc0fbbd0 msgr2=0x7f8afc0fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:48:50.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.243+0000 7f8b0094b700 1 -- 172.21.15.43:0/2499553470 shutdown_connections 2024-03-21T12:48:50.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.243+0000 7f8b0094b700 1 -- 172.21.15.43:0/2499553470 wait complete. 2024-03-21T12:48:50.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.243+0000 7f8b0094b700 1 Processor -- start 2024-03-21T12:48:50.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.243+0000 7f8b0094b700 1 -- start start 2024-03-21T12:48:50.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.243+0000 7f8b0094b700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8afc1a8ae0 con 0x7f8afc104a60 2024-03-21T12:48:50.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.243+0000 7f8b0094b700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8afc1a9d00 con 0x7f8afc100630 2024-03-21T12:48:50.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.243+0000 7f8b0094b700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8afc1aaf40 con 0x7f8afc1087a0 2024-03-21T12:48:50.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.244+0000 7f8af9d9b700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8afc104a60 0x7f8afc1a3c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37660/0 (socket says 172.21.15.43:37660) 2024-03-21T12:48:50.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.244+0000 7f8af9d9b700 1 -- 172.21.15.43:0/3766312796 learned_addr learned my addr 172.21.15.43:0/3766312796 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:48:50.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.244+0000 7f8aeb7fe700 1 -- 172.21.15.43:0/3766312796 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 155320263 0 0) 0x7f8afc1aaf40 con 0x7f8afc1087a0 2024-03-21T12:48:50.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.244+0000 7f8aeb7fe700 1 -- 172.21.15.43:0/3766312796 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ae40061c0 con 0x7f8afc1087a0 2024-03-21T12:48:50.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.244+0000 7f8aeb7fe700 1 -- 172.21.15.43:0/3766312796 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3111047127 0 0) 0x7f8afc1a8ae0 con 0x7f8afc104a60 2024-03-21T12:48:50.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.244+0000 7f8aeb7fe700 1 -- 172.21.15.43:0/3766312796 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8afc1aaf40 con 0x7f8afc104a60 2024-03-21T12:48:50.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.244+0000 7f8aeb7fe700 1 -- 172.21.15.43:0/3766312796 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 667613008 0 0) 0x7f8afc1a9d00 con 0x7f8afc100630 2024-03-21T12:48:50.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.244+0000 7f8aeb7fe700 1 -- 172.21.15.43:0/3766312796 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8afc1a8ae0 con 0x7f8afc100630 2024-03-21T12:48:50.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.244+0000 7f8aeb7fe700 1 -- 172.21.15.43:0/3766312796 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2786931425 0 0) 0x7f8ae40061c0 con 0x7f8afc1087a0 2024-03-21T12:48:50.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.244+0000 7f8aeb7fe700 1 -- 172.21.15.43:0/3766312796 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8afc1a9d00 con 0x7f8afc1087a0 2024-03-21T12:48:50.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.244+0000 7f8aeb7fe700 1 -- 172.21.15.43:0/3766312796 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f8af4003b00 con 0x7f8afc1087a0 2024-03-21T12:48:50.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.244+0000 7f8aeb7fe700 1 -- 172.21.15.43:0/3766312796 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1270907812 0 0) 0x7f8afc1aaf40 con 0x7f8afc104a60 2024-03-21T12:48:50.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.244+0000 7f8aeb7fe700 1 -- 172.21.15.43:0/3766312796 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8ae40061c0 con 0x7f8afc104a60 2024-03-21T12:48:50.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.244+0000 7f8aeb7fe700 1 -- 172.21.15.43:0/3766312796 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f8af0001a30 con 0x7f8afc104a60 2024-03-21T12:48:50.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.244+0000 7f8aeb7fe700 1 -- 172.21.15.43:0/3766312796 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 776930577 0 0) 0x7f8afc1a8ae0 con 0x7f8afc100630 2024-03-21T12:48:50.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.244+0000 7f8aeb7fe700 1 -- 172.21.15.43:0/3766312796 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8afc1aaf40 con 0x7f8afc100630 2024-03-21T12:48:50.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.244+0000 7f8aeb7fe700 1 -- 172.21.15.43:0/3766312796 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f8aec001830 con 0x7f8afc100630 2024-03-21T12:48:50.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.244+0000 7f8aeb7fe700 1 -- 172.21.15.43:0/3766312796 <== mon.2 v1:172.21.15.43:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3136967247 0 0) 0x7f8afc1a9d00 con 0x7f8afc1087a0 2024-03-21T12:48:50.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.244+0000 7f8aeb7fe700 1 -- 172.21.15.43:0/3766312796 >> v1:172.21.15.155:6789/0 conn(0x7f8afc100630 legacy=0x7f8afc106930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:48:50.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.244+0000 7f8aeb7fe700 1 -- 172.21.15.43:0/3766312796 >> v1:172.21.15.43:6789/0 conn(0x7f8afc104a60 legacy=0x7f8afc1a3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:48:50.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.244+0000 7f8aeb7fe700 1 -- 172.21.15.43:0/3766312796 --> v1:172.21.15.43:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8afc1ac180 con 0x7f8afc1087a0 2024-03-21T12:48:50.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.244+0000 7f8b0094b700 1 -- 172.21.15.43:0/3766312796 --> v1:172.21.15.43:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8afc1ab170 con 0x7f8afc1087a0 2024-03-21T12:48:50.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.244+0000 7f8b0094b700 1 -- 172.21.15.43:0/3766312796 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8afc1ab6b0 con 0x7f8afc1087a0 2024-03-21T12:48:50.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.245+0000 7f8aeb7fe700 1 -- 172.21.15.43:0/3766312796 <== mon.2 v1:172.21.15.43:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f8af4002090 con 0x7f8afc1087a0 2024-03-21T12:48:50.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.245+0000 7f8aeb7fe700 1 -- 172.21.15.43:0/3766312796 <== mon.2 v1:172.21.15.43:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f8af4005040 con 0x7f8afc1087a0 2024-03-21T12:48:50.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.245+0000 7f8b0094b700 1 -- 172.21.15.43:0/3766312796 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8adc005150 con 0x7f8afc1087a0 2024-03-21T12:48:50.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.246+0000 7f8aeb7fe700 1 -- 172.21.15.43:0/3766312796 <== mon.2 v1:172.21.15.43:6790/0 7 ==== mgrmap(e 14) v1 ==== 92039+0+0 (unknown 3498086941 0 0) 0x7f8af401b7d0 con 0x7f8afc1087a0 2024-03-21T12:48:50.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.246+0000 7f8aeb7fe700 1 -- 172.21.15.43:0/3766312796 <== mon.2 v1:172.21.15.43:6790/0 8 ==== osd_map(14..14 src has 1..14) v4 ==== 1833+0+0 (unknown 140055632 0 0) 0x7f8af4087720 con 0x7f8afc1087a0 2024-03-21T12:48:50.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.251+0000 7f8aeb7fe700 1 -- 172.21.15.43:0/3766312796 <== mon.2 v1:172.21.15.43:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f8af4055f20 con 0x7f8afc1087a0 2024-03-21T12:48:50.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:48:50.468+0000 7f8b0094b700 1 -- 172.21.15.43:0/3766312796 --> v1:172.21.15.43:6800/4081968544 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f8adc000bc0 con 0x7f8ae406f150 2024-03-21T12:48:50.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:50 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T12:48:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:50 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T12:48:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:50 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:48:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:50 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T12:48:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:50 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T12:48:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:50 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:48:50.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:50 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T12:48:50.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:50 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T12:48:50.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:50 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:48:51.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:51 smithi043 ceph-mon[30062]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:48:51.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:51 smithi043 ceph-mon[30062]: from='client.34169 v1:172.21.15.43:0/3766312796' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:48:51.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:51 smithi043 ceph-mon[41699]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:48:51.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:51 smithi043 ceph-mon[41699]: from='client.34169 v1:172.21.15.43:0/3766312796' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:48:51.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:51 smithi155 ceph-mon[33846]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:48:51.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:51 smithi155 ceph-mon[33846]: from='client.34169 v1:172.21.15.43:0/3766312796' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:48:53.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:53 smithi155 ceph-mon[33846]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:48:53.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:53 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3973965202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63230fe0-2225-4ebd-a6de-aa55fc7ea0b5"}]: dispatch 2024-03-21T12:48:53.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:53 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3973965202' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63230fe0-2225-4ebd-a6de-aa55fc7ea0b5"}]': finished 2024-03-21T12:48:53.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:53 smithi155 ceph-mon[33846]: osdmap e15: 3 total, 2 up, 3 in 2024-03-21T12:48:53.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:53 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T12:48:53.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:53 smithi043 ceph-mon[30062]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:48:53.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:53 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3973965202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63230fe0-2225-4ebd-a6de-aa55fc7ea0b5"}]: dispatch 2024-03-21T12:48:53.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:53 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3973965202' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63230fe0-2225-4ebd-a6de-aa55fc7ea0b5"}]': finished 2024-03-21T12:48:53.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:53 smithi043 ceph-mon[30062]: osdmap e15: 3 total, 2 up, 3 in 2024-03-21T12:48:53.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:53 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T12:48:53.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:53 smithi043 ceph-mon[41699]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:48:53.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:53 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3973965202' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63230fe0-2225-4ebd-a6de-aa55fc7ea0b5"}]: dispatch 2024-03-21T12:48:53.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:53 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3973965202' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63230fe0-2225-4ebd-a6de-aa55fc7ea0b5"}]': finished 2024-03-21T12:48:53.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:53 smithi043 ceph-mon[41699]: osdmap e15: 3 total, 2 up, 3 in 2024-03-21T12:48:53.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:53 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T12:48:54.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:54 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1815435298' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T12:48:54.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:54 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1815435298' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T12:48:54.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:54 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1815435298' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T12:48:55.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:55 smithi155 ceph-mon[33846]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:48:55.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:55 smithi043 ceph-mon[30062]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:48:55.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:55 smithi043 ceph-mon[41699]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:48:57.804 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:57 smithi043 ceph-mon[30062]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:48:57.804 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:57 smithi043 ceph-mon[41699]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:48:57.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:57 smithi155 ceph-mon[33846]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:48:59.839 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:48:59 smithi043 ceph-mon[30062]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:48:59.839 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:48:59 smithi043 ceph-mon[41699]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:48:59.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:48:59 smithi155 ceph-mon[33846]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:49:00.784 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:00 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T12:49:00.784 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:00 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:00.784 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:00 smithi043 ceph-mon[30062]: Deploying daemon osd.2 on smithi043 2024-03-21T12:49:00.784 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:00 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T12:49:00.784 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:00 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:00.784 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:00 smithi043 ceph-mon[41699]: Deploying daemon osd.2 on smithi043 2024-03-21T12:49:00.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:00 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T12:49:00.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:00 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:00.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:00 smithi155 ceph-mon[33846]: Deploying daemon osd.2 on smithi043 2024-03-21T12:49:01.841 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:01 smithi043 ceph-mon[30062]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:49:01.841 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:01 smithi043 ceph-mon[41699]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:49:01.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:01 smithi155 ceph-mon[33846]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:49:03.824 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:03 smithi043 ceph-mon[30062]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:49:03.824 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:03 smithi043 ceph-mon[41699]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:49:03.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:03 smithi155 ceph-mon[33846]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:49:05.876 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:05 smithi043 ceph-mon[30062]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:49:05.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:05 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:05.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:05 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:05.877 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:05 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:49:05.877 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:05 smithi043 ceph-mon[41699]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:49:05.877 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:05 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:05.877 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:05 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:05.877 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:05 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:49:05.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:05 smithi155 ceph-mon[33846]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:49:05.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:05 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:05.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:05 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:05.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:05 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:49:07.617 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:07 smithi043 ceph-mon[30062]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:49:07.617 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:07 smithi043 ceph-mon[41699]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:49:07.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:07 smithi155 ceph-mon[33846]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:49:08.884 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:08 smithi043 ceph-mon[30062]: from='osd.2 v1:172.21.15.43:6809/1358463812' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T12:49:08.884 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:08 smithi043 ceph-mon[41699]: from='osd.2 v1:172.21.15.43:6809/1358463812' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T12:49:08.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:08 smithi155 ceph-mon[33846]: from='osd.2 v1:172.21.15.43:6809/1358463812' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T12:49:08.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:08.971+0000 7f8aeb7fe700 1 -- 172.21.15.43:0/3766312796 <== mgr.14152 v1:172.21.15.43:6800/4081968544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 901172121) 0x7f8adc000bc0 con 0x7f8ae406f150 2024-03-21T12:49:08.971 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 2 on host 'smithi043' 2024-03-21T12:49:08.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:08.973+0000 7f8b0094b700 1 -- 172.21.15.43:0/3766312796 >> v1:172.21.15.43:6800/4081968544 conn(0x7f8ae406f150 legacy=0x7f8ae4071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:49:08.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:08.973+0000 7f8b0094b700 1 -- 172.21.15.43:0/3766312796 >> v1:172.21.15.43:6790/0 conn(0x7f8afc1087a0 legacy=0x7f8afc1a73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:49:08.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:08.973+0000 7f8b0094b700 1 -- 172.21.15.43:0/3766312796 shutdown_connections 2024-03-21T12:49:08.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:08.973+0000 7f8b0094b700 1 -- 172.21.15.43:0/3766312796 >> 172.21.15.43:0/3766312796 conn(0x7f8afc0fbbd0 msgr2=0x7f8afc107e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:49:08.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:08.973+0000 7f8b0094b700 1 -- 172.21.15.43:0/3766312796 shutdown_connections 2024-03-21T12:49:08.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:08.973+0000 7f8b0094b700 1 -- 172.21.15.43:0/3766312796 wait complete. 2024-03-21T12:49:09.532 DEBUG:teuthology.orchestra.run.smithi043:osd.2> sudo journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@osd.2.service 2024-03-21T12:49:09.535 INFO:tasks.cephadm:Deploying osd.3 on smithi043 with /dev/vg_nvme/lv_1... 2024-03-21T12:49:09.535 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-21T12:49:09.848 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:49:09.887 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:09 smithi043 ceph-mon[30062]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:49:09.887 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:09 smithi043 ceph-mon[30062]: from='osd.2 v1:172.21.15.43:6809/1358463812' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-21T12:49:09.887 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:09 smithi043 ceph-mon[30062]: osdmap e16: 3 total, 2 up, 3 in 2024-03-21T12:49:09.887 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:09 smithi043 ceph-mon[30062]: from='osd.2 v1:172.21.15.43:6809/1358463812' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-21T12:49:09.887 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:09 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T12:49:09.887 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:09 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:09.888 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:09 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:09.888 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:09 smithi043 ceph-mon[41699]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:49:09.888 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:09 smithi043 ceph-mon[41699]: from='osd.2 v1:172.21.15.43:6809/1358463812' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-21T12:49:09.888 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:09 smithi043 ceph-mon[41699]: osdmap e16: 3 total, 2 up, 3 in 2024-03-21T12:49:09.888 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:09 smithi043 ceph-mon[41699]: from='osd.2 v1:172.21.15.43:6809/1358463812' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-21T12:49:09.888 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:09 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T12:49:09.888 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:09 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:09.888 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:09 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:09 smithi155 ceph-mon[33846]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:49:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:09 smithi155 ceph-mon[33846]: from='osd.2 v1:172.21.15.43:6809/1358463812' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-21T12:49:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:09 smithi155 ceph-mon[33846]: osdmap e16: 3 total, 2 up, 3 in 2024-03-21T12:49:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:09 smithi155 ceph-mon[33846]: from='osd.2 v1:172.21.15.43:6809/1358463812' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-21T12:49:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:09 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T12:49:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:09 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:09 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:10.234 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 12:49:09 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-2[58226]: 2024-03-21T12:49:09.886+0000 7f6a3f1df700 -1 osd.2 0 waiting for initial osdmap 2024-03-21T12:49:10.234 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 12:49:09 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-2[58226]: 2024-03-21T12:49:09.889+0000 7f6a36fca700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T12:49:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:10 smithi155 ceph-mon[33846]: from='osd.2 v1:172.21.15.43:6809/1358463812' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-03-21T12:49:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:10 smithi155 ceph-mon[33846]: osdmap e17: 3 total, 2 up, 3 in 2024-03-21T12:49:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:10 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T12:49:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:10 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T12:49:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:10 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:10 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:10 smithi155 ceph-mon[33846]: from='osd.2 v1:172.21.15.43:6809/1358463812' entity='osd.2' 2024-03-21T12:49:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:10 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T12:49:10.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:10 smithi043 ceph-mon[30062]: from='osd.2 v1:172.21.15.43:6809/1358463812' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-03-21T12:49:10.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:10 smithi043 ceph-mon[30062]: osdmap e17: 3 total, 2 up, 3 in 2024-03-21T12:49:10.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:10 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T12:49:10.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:10 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T12:49:10.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:10 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:10.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:10 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:10.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:10 smithi043 ceph-mon[30062]: from='osd.2 v1:172.21.15.43:6809/1358463812' entity='osd.2' 2024-03-21T12:49:10.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:10 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T12:49:10.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:10 smithi043 ceph-mon[41699]: from='osd.2 v1:172.21.15.43:6809/1358463812' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-03-21T12:49:10.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:10 smithi043 ceph-mon[41699]: osdmap e17: 3 total, 2 up, 3 in 2024-03-21T12:49:10.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:10 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T12:49:10.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:10 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T12:49:10.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:10 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:10.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:10 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:10.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:10 smithi043 ceph-mon[41699]: from='osd.2 v1:172.21.15.43:6809/1358463812' entity='osd.2' 2024-03-21T12:49:10.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:10 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T12:49:11.879 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-21T12:49:11.880 INFO:teuthology.orchestra.run.smithi043.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-21T12:49:11.880 INFO:teuthology.orchestra.run.smithi043.stderr: stderr: 10+0 records in 2024-03-21T12:49:11.880 INFO:teuthology.orchestra.run.smithi043.stderr:10+0 records out 2024-03-21T12:49:11.880 INFO:teuthology.orchestra.run.smithi043.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00941808 s, 1.1 GB/s 2024-03-21T12:49:11.880 INFO:teuthology.orchestra.run.smithi043.stderr:--> Zapping successful for: 2024-03-21T12:49:11.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:11 smithi043 ceph-mon[30062]: purged_snaps scrub starts 2024-03-21T12:49:11.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:11 smithi043 ceph-mon[30062]: purged_snaps scrub ok 2024-03-21T12:49:11.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:11 smithi043 ceph-mon[30062]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:49:11.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:11 smithi043 ceph-mon[30062]: osd.2 v1:172.21.15.43:6809/1358463812 boot 2024-03-21T12:49:11.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:11 smithi043 ceph-mon[30062]: osdmap e18: 3 total, 3 up, 3 in 2024-03-21T12:49:11.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:11 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T12:49:11.960 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:11 smithi043 ceph-mon[41699]: purged_snaps scrub starts 2024-03-21T12:49:11.960 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:11 smithi043 ceph-mon[41699]: purged_snaps scrub ok 2024-03-21T12:49:11.960 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:11 smithi043 ceph-mon[41699]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:49:11.960 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:11 smithi043 ceph-mon[41699]: osd.2 v1:172.21.15.43:6809/1358463812 boot 2024-03-21T12:49:11.960 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:11 smithi043 ceph-mon[41699]: osdmap e18: 3 total, 3 up, 3 in 2024-03-21T12:49:11.960 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:11 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T12:49:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:11 smithi155 ceph-mon[33846]: purged_snaps scrub starts 2024-03-21T12:49:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:11 smithi155 ceph-mon[33846]: purged_snaps scrub ok 2024-03-21T12:49:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:11 smithi155 ceph-mon[33846]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-03-21T12:49:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:11 smithi155 ceph-mon[33846]: osd.2 v1:172.21.15.43:6809/1358463812 boot 2024-03-21T12:49:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:11 smithi155 ceph-mon[33846]: osdmap e18: 3 total, 3 up, 3 in 2024-03-21T12:49:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:11 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T12:49:13.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:12 smithi155 ceph-mon[33846]: osdmap e19: 3 total, 3 up, 3 in 2024-03-21T12:49:13.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:12 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:49:13.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:12 smithi043 ceph-mon[30062]: osdmap e19: 3 total, 3 up, 3 in 2024-03-21T12:49:13.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:12 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:49:13.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:12 smithi043 ceph-mon[41699]: osdmap e19: 3 total, 3 up, 3 in 2024-03-21T12:49:13.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:12 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:49:13.309 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi043:vg_nvme/lv_1 2024-03-21T12:49:13.896 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:49:14.173 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:13 smithi043 ceph-mon[30062]: pgmap v79: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T12:49:14.173 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:13 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-21T12:49:14.173 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:13 smithi043 ceph-mon[30062]: osdmap e20: 3 total, 3 up, 3 in 2024-03-21T12:49:14.173 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:13 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:49:14.173 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:13 smithi043 ceph-mon[41699]: pgmap v79: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T12:49:14.173 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:13 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-21T12:49:14.174 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:13 smithi043 ceph-mon[41699]: osdmap e20: 3 total, 3 up, 3 in 2024-03-21T12:49:14.174 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:13 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:49:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:13 smithi155 ceph-mon[33846]: pgmap v79: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T12:49:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:13 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-21T12:49:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:13 smithi155 ceph-mon[33846]: osdmap e20: 3 total, 3 up, 3 in 2024-03-21T12:49:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:13 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:49:14.440 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 21 12:49:14 smithi043 sudo[61784]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T12:49:14.440 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 21 12:49:14 smithi043 sudo[61784]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T12:49:14.440 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 21 12:49:14 smithi043 sudo[61784]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T12:49:14.440 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 21 12:49:14 smithi043 sudo[61784]: pam_unix(sudo:session): session closed for user root 2024-03-21T12:49:14.693 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 21 12:49:14 smithi043 sudo[62311]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T12:49:14.693 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 21 12:49:14 smithi043 sudo[62311]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T12:49:14.693 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 21 12:49:14 smithi043 sudo[62311]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T12:49:14.694 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 21 12:49:14 smithi043 sudo[62311]: pam_unix(sudo:session): session closed for user root 2024-03-21T12:49:14.952 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:14 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-21T12:49:14.952 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:14 smithi043 ceph-mon[30062]: osdmap e21: 3 total, 3 up, 3 in 2024-03-21T12:49:14.952 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:14 smithi043 ceph-mon[30062]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T12:49:14.953 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:14 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:14.953 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:14 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-21T12:49:14.953 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:14 smithi043 ceph-mon[41699]: osdmap e21: 3 total, 3 up, 3 in 2024-03-21T12:49:14.953 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:14 smithi043 ceph-mon[41699]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T12:49:14.953 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:14 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:15.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:14 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-21T12:49:15.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:14 smithi155 ceph-mon[33846]: osdmap e21: 3 total, 3 up, 3 in 2024-03-21T12:49:15.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:14 smithi155 ceph-mon[33846]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-03-21T12:49:15.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:14 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:15.221 INFO:journalctl@ceph.osd.1.smithi043.stdout:Mar 21 12:49:14 smithi043 sudo[62321]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T12:49:15.221 INFO:journalctl@ceph.osd.1.smithi043.stdout:Mar 21 12:49:14 smithi043 sudo[62321]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T12:49:15.221 INFO:journalctl@ceph.osd.1.smithi043.stdout:Mar 21 12:49:14 smithi043 sudo[62321]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T12:49:15.221 INFO:journalctl@ceph.osd.1.smithi043.stdout:Mar 21 12:49:14 smithi043 sudo[62321]: pam_unix(sudo:session): session closed for user root 2024-03-21T12:49:15.483 INFO:journalctl@ceph.osd.1.smithi043.stdout:Mar 21 12:49:15 smithi043 sudo[62324]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T12:49:15.484 INFO:journalctl@ceph.osd.1.smithi043.stdout:Mar 21 12:49:15 smithi043 sudo[62324]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T12:49:15.484 INFO:journalctl@ceph.osd.1.smithi043.stdout:Mar 21 12:49:15 smithi043 sudo[62324]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T12:49:15.484 INFO:journalctl@ceph.osd.1.smithi043.stdout:Mar 21 12:49:15 smithi043 sudo[62324]: pam_unix(sudo:session): session closed for user root 2024-03-21T12:49:15.739 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 12:49:15 smithi043 sudo[62383]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T12:49:15.740 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 12:49:15 smithi043 sudo[62383]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T12:49:15.740 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 12:49:15 smithi043 sudo[62383]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T12:49:15.740 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 12:49:15 smithi043 sudo[62383]: pam_unix(sudo:session): session closed for user root 2024-03-21T12:49:16.019 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 12:49:15 smithi043 sudo[62403]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T12:49:16.019 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 12:49:15 smithi043 sudo[62403]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T12:49:16.019 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 12:49:15 smithi043 sudo[62403]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T12:49:16.019 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 12:49:15 smithi043 sudo[62403]: pam_unix(sudo:session): session closed for user root 2024-03-21T12:49:16.290 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 sudo[62424]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T12:49:16.290 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 sudo[62424]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T12:49:16.290 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 sudo[62424]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T12:49:16.290 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 sudo[62424]: pam_unix(sudo:session): session closed for user root 2024-03-21T12:49:16.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.319+0000 7f0f6100f700 1 -- 172.21.15.43:0/3994278246 >> v1:172.21.15.43:6789/0 conn(0x7f0f5c100e80 legacy=0x7f0f5c1012a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:49:16.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.319+0000 7f0f6100f700 1 -- 172.21.15.43:0/3994278246 shutdown_connections 2024-03-21T12:49:16.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.319+0000 7f0f6100f700 1 -- 172.21.15.43:0/3994278246 >> 172.21.15.43:0/3994278246 conn(0x7f0f5c0fc420 msgr2=0x7f0f5c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:49:16.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.319+0000 7f0f6100f700 1 -- 172.21.15.43:0/3994278246 shutdown_connections 2024-03-21T12:49:16.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.319+0000 7f0f6100f700 1 -- 172.21.15.43:0/3994278246 wait complete. 2024-03-21T12:49:16.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.320+0000 7f0f6100f700 1 Processor -- start 2024-03-21T12:49:16.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.320+0000 7f0f6100f700 1 -- start start 2024-03-21T12:49:16.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.320+0000 7f0f6100f700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0f5c06aa30 con 0x7f0f5c1052a0 2024-03-21T12:49:16.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.320+0000 7f0f6100f700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0f5c1a3820 con 0x7f0f5c100e80 2024-03-21T12:49:16.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.320+0000 7f0f6100f700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0f5c1a4a60 con 0x7f0f5c108f30 2024-03-21T12:49:16.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.320+0000 7f0f5a59c700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0f5c1052a0 0x7f0f5c19d980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52324/0 (socket says 172.21.15.43:52324) 2024-03-21T12:49:16.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.320+0000 7f0f5b59e700 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7f0f5c108f30 0x7f0f5c1a10b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:34106/0 (socket says 172.21.15.43:34106) 2024-03-21T12:49:16.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.320+0000 7f0f5a59c700 1 -- 172.21.15.43:0/3733671395 learned_addr learned my addr 172.21.15.43:0/3733671395 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:49:16.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.320+0000 7f0f53fff700 1 -- 172.21.15.43:0/3733671395 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 420978891 0 0) 0x7f0f5c1a4a60 con 0x7f0f5c108f30 2024-03-21T12:49:16.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.320+0000 7f0f53fff700 1 -- 172.21.15.43:0/3733671395 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f480035b0 con 0x7f0f5c108f30 2024-03-21T12:49:16.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.320+0000 7f0f53fff700 1 -- 172.21.15.43:0/3733671395 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2032020057 0 0) 0x7f0f5c06aa30 con 0x7f0f5c1052a0 2024-03-21T12:49:16.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.320+0000 7f0f53fff700 1 -- 172.21.15.43:0/3733671395 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f5c1a4a60 con 0x7f0f5c1052a0 2024-03-21T12:49:16.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.321+0000 7f0f53fff700 1 -- 172.21.15.43:0/3733671395 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4088836450 0 0) 0x7f0f480035b0 con 0x7f0f5c108f30 2024-03-21T12:49:16.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.321+0000 7f0f53fff700 1 -- 172.21.15.43:0/3733671395 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0f5c06aa30 con 0x7f0f5c108f30 2024-03-21T12:49:16.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.321+0000 7f0f53fff700 1 -- 172.21.15.43:0/3733671395 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1239098437 0 0) 0x7f0f5c1a4a60 con 0x7f0f5c1052a0 2024-03-21T12:49:16.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.321+0000 7f0f53fff700 1 -- 172.21.15.43:0/3733671395 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0f480035b0 con 0x7f0f5c1052a0 2024-03-21T12:49:16.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.321+0000 7f0f53fff700 1 -- 172.21.15.43:0/3733671395 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f0f540019c0 con 0x7f0f5c108f30 2024-03-21T12:49:16.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.321+0000 7f0f53fff700 1 -- 172.21.15.43:0/3733671395 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f0f440017b0 con 0x7f0f5c1052a0 2024-03-21T12:49:16.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.321+0000 7f0f53fff700 1 -- 172.21.15.43:0/3733671395 <== mon.2 v1:172.21.15.43:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3878559392 0 0) 0x7f0f5c06aa30 con 0x7f0f5c108f30 2024-03-21T12:49:16.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.321+0000 7f0f53fff700 1 -- 172.21.15.43:0/3733671395 >> v1:172.21.15.155:6789/0 conn(0x7f0f5c100e80 legacy=0x7f0f5c19a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:49:16.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.321+0000 7f0f53fff700 1 -- 172.21.15.43:0/3733671395 >> v1:172.21.15.43:6789/0 conn(0x7f0f5c1052a0 legacy=0x7f0f5c19d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:49:16.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.321+0000 7f0f53fff700 1 -- 172.21.15.43:0/3733671395 --> v1:172.21.15.43:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5c1a5ca0 con 0x7f0f5c108f30 2024-03-21T12:49:16.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.321+0000 7f0f6100f700 1 -- 172.21.15.43:0/3733671395 --> v1:172.21.15.43:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5c1a2830 con 0x7f0f5c108f30 2024-03-21T12:49:16.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.321+0000 7f0f6100f700 1 -- 172.21.15.43:0/3733671395 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5c1a2d70 con 0x7f0f5c108f30 2024-03-21T12:49:16.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.321+0000 7f0f53fff700 1 -- 172.21.15.43:0/3733671395 <== mon.2 v1:172.21.15.43:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f0f54000f10 con 0x7f0f5c108f30 2024-03-21T12:49:16.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.321+0000 7f0f53fff700 1 -- 172.21.15.43:0/3733671395 <== mon.2 v1:172.21.15.43:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f0f54005bc0 con 0x7f0f5c108f30 2024-03-21T12:49:16.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.322+0000 7f0f6100f700 1 -- 172.21.15.43:0/3733671395 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f3c005150 con 0x7f0f5c108f30 2024-03-21T12:49:16.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.322+0000 7f0f53fff700 1 -- 172.21.15.43:0/3733671395 <== mon.2 v1:172.21.15.43:6790/0 7 ==== mgrmap(e 15) v1 ==== 92095+0+0 (unknown 2546926431 0 0) 0x7f0f54003680 con 0x7f0f5c108f30 2024-03-21T12:49:16.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.323+0000 7f0f53fff700 1 -- 172.21.15.43:0/3733671395 <== mon.2 v1:172.21.15.43:6790/0 8 ==== osd_map(22..22 src has 1..22) v4 ==== 2511+0+0 (unknown 3403954633 0 0) 0x7f0f54088760 con 0x7f0f5c108f30 2024-03-21T12:49:16.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.328+0000 7f0f53fff700 1 -- 172.21.15.43:0/3733671395 <== mon.2 v1:172.21.15.43:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f0f54056c40 con 0x7f0f5c108f30 2024-03-21T12:49:16.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:16.542+0000 7f0f6100f700 1 -- 172.21.15.43:0/3733671395 --> v1:172.21.15.43:6800/4081968544 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f3c000bc0 con 0x7f0f4806f020 2024-03-21T12:49:16.544 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[41699]: osdmap e22: 3 total, 3 up, 3 in 2024-03-21T12:49:16.544 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:16.544 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:16.544 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:49:16.544 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:49:16.544 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:49:16.544 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:16.544 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:16.544 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:49:16.544 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:16.544 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:49:16.544 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:16.545 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:49:16.545 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:16.545 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[41699]: mgrmap e15: y(active, since 2m), standbys: x 2024-03-21T12:49:16.545 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T12:49:16.545 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:49:16.545 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:49:16.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 sudo[62453]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T12:49:16.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 sudo[62453]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T12:49:16.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 sudo[62453]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T12:49:16.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 sudo[62453]: pam_unix(sudo:session): session closed for user root 2024-03-21T12:49:16.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[30062]: osdmap e22: 3 total, 3 up, 3 in 2024-03-21T12:49:16.545 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:16.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:16.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:49:16.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:49:16.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:49:16.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:16.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:16.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:49:16.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:16.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:49:16.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:16.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:49:16.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:16.546 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[30062]: mgrmap e15: y(active, since 2m), standbys: x 2024-03-21T12:49:16.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T12:49:16.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:49:16.547 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:16 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:49:16.556 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 ceph-mon[33846]: osdmap e22: 3 total, 3 up, 3 in 2024-03-21T12:49:16.556 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:16.556 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:16.556 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:49:16.556 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:49:16.556 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:49:16.557 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:16.557 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:16.557 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:49:16.557 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:16.557 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:49:16.557 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:16.557 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:49:16.557 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:16.557 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 ceph-mon[33846]: mgrmap e15: y(active, since 2m), standbys: x 2024-03-21T12:49:16.557 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T12:49:16.557 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:49:16.557 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:49:16.557 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 sudo[36671]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T12:49:16.557 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 sudo[36671]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T12:49:16.557 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 sudo[36671]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T12:49:16.558 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 sudo[36671]: pam_unix(sudo:session): session closed for user root 2024-03-21T12:49:16.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 sudo[36674]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T12:49:16.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 sudo[36674]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T12:49:16.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 sudo[36674]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T12:49:16.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:16 smithi155 sudo[36674]: pam_unix(sudo:session): session closed for user root 2024-03-21T12:49:17.354 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 sudo[62598]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T12:49:17.355 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 sudo[62598]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T12:49:17.355 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 sudo[62598]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T12:49:17.355 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 sudo[62598]: pam_unix(sudo:session): session closed for user root 2024-03-21T12:49:17.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:17 smithi155 ceph-mon[33846]: Detected new or changed devices on smithi043 2024-03-21T12:49:17.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:17 smithi155 ceph-mon[33846]: Adjusting osd_memory_target on smithi043 to 5356M 2024-03-21T12:49:17.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:17 smithi155 ceph-mon[33846]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T12:49:17.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:17 smithi155 ceph-mon[33846]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-03-21T12:49:17.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:17 smithi155 ceph-mon[33846]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T12:49:17.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:17 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T12:49:17.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:17 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:49:17.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:17 smithi155 ceph-mon[33846]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T12:49:17.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:17 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:49:17.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:17 smithi155 ceph-mon[33846]: from='client.34202 v1:172.21.15.43:0/3733671395' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:49:17.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:17 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T12:49:17.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:17 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T12:49:17.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:17 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:17.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:17 smithi155 ceph-mon[33846]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T12:49:17.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:17 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:17.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:17 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T12:49:17.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:17 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:49:17.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:17 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:49:17.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 sudo[62623]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T12:49:17.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 sudo[62623]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T12:49:17.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 sudo[62623]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T12:49:17.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 sudo[62623]: pam_unix(sudo:session): session closed for user root 2024-03-21T12:49:17.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[41699]: Detected new or changed devices on smithi043 2024-03-21T12:49:17.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[41699]: Adjusting osd_memory_target on smithi043 to 5356M 2024-03-21T12:49:17.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[41699]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T12:49:17.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[41699]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-03-21T12:49:17.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[41699]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T12:49:17.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T12:49:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:49:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[41699]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T12:49:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:49:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[41699]: from='client.34202 v1:172.21.15.43:0/3733671395' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:49:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T12:49:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T12:49:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[41699]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T12:49:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T12:49:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:49:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:49:17.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[30062]: Detected new or changed devices on smithi043 2024-03-21T12:49:17.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[30062]: Adjusting osd_memory_target on smithi043 to 5356M 2024-03-21T12:49:17.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[30062]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T12:49:17.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[30062]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 89 MiB used, 268 GiB / 268 GiB avail 2024-03-21T12:49:17.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[30062]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T12:49:17.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T12:49:17.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:49:17.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[30062]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T12:49:17.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:49:17.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[30062]: from='client.34202 v1:172.21.15.43:0/3733671395' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi043:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:49:17.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T12:49:17.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T12:49:17.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:17.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[30062]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T12:49:17.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:17.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T12:49:17.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:49:17.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:17 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:49:18.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:18 smithi043 ceph-mon[30062]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T12:49:18.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:18 smithi043 ceph-mon[30062]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T12:49:18.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:18 smithi043 ceph-mon[41699]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T12:49:18.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:18 smithi043 ceph-mon[41699]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T12:49:18.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:18 smithi155 ceph-mon[33846]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T12:49:18.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:18 smithi155 ceph-mon[33846]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T12:49:19.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:19 smithi155 ceph-mon[33846]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-21T12:49:19.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:19 smithi043 ceph-mon[30062]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-21T12:49:19.733 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:19 smithi043 ceph-mon[41699]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-21T12:49:20.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:20 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2638477548' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac04f1e6-c31c-44b0-909d-a8679d39b335"}]: dispatch 2024-03-21T12:49:20.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:20 smithi155 ceph-mon[33846]: from='client.34211 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac04f1e6-c31c-44b0-909d-a8679d39b335"}]: dispatch 2024-03-21T12:49:20.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:20 smithi155 ceph-mon[33846]: from='client.34211 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac04f1e6-c31c-44b0-909d-a8679d39b335"}]': finished 2024-03-21T12:49:20.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:20 smithi155 ceph-mon[33846]: osdmap e23: 4 total, 3 up, 4 in 2024-03-21T12:49:20.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:20 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T12:49:20.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:20 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2672669567' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T12:49:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:20 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2638477548' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac04f1e6-c31c-44b0-909d-a8679d39b335"}]: dispatch 2024-03-21T12:49:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:20 smithi043 ceph-mon[30062]: from='client.34211 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac04f1e6-c31c-44b0-909d-a8679d39b335"}]: dispatch 2024-03-21T12:49:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:20 smithi043 ceph-mon[30062]: from='client.34211 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac04f1e6-c31c-44b0-909d-a8679d39b335"}]': finished 2024-03-21T12:49:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:20 smithi043 ceph-mon[30062]: osdmap e23: 4 total, 3 up, 4 in 2024-03-21T12:49:20.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:20 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T12:49:20.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:20 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2672669567' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T12:49:20.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:20 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2638477548' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac04f1e6-c31c-44b0-909d-a8679d39b335"}]: dispatch 2024-03-21T12:49:20.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:20 smithi043 ceph-mon[41699]: from='client.34211 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ac04f1e6-c31c-44b0-909d-a8679d39b335"}]: dispatch 2024-03-21T12:49:20.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:20 smithi043 ceph-mon[41699]: from='client.34211 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ac04f1e6-c31c-44b0-909d-a8679d39b335"}]': finished 2024-03-21T12:49:20.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:20 smithi043 ceph-mon[41699]: osdmap e23: 4 total, 3 up, 4 in 2024-03-21T12:49:20.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:20 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T12:49:20.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:20 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2672669567' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T12:49:21.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:21 smithi155 ceph-mon[33846]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-21T12:49:21.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:21 smithi043 ceph-mon[30062]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-21T12:49:21.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:21 smithi043 ceph-mon[41699]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-21T12:49:23.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:23 smithi155 ceph-mon[33846]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-03-21T12:49:23.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:23 smithi043 ceph-mon[30062]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-03-21T12:49:23.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:23 smithi043 ceph-mon[41699]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-03-21T12:49:25.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:25 smithi155 ceph-mon[33846]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-21T12:49:25.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:25 smithi043 ceph-mon[30062]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-21T12:49:25.733 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:25 smithi043 ceph-mon[41699]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-21T12:49:26.483 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:26 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T12:49:26.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:26 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:26.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:26 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T12:49:26.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:26 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:26.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:26 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T12:49:26.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:26 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:27.686 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:27 smithi043 ceph-mon[30062]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-21T12:49:27.686 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:27 smithi043 ceph-mon[30062]: Deploying daemon osd.3 on smithi043 2024-03-21T12:49:27.687 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:27 smithi043 ceph-mon[41699]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-21T12:49:27.687 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:27 smithi043 ceph-mon[41699]: Deploying daemon osd.3 on smithi043 2024-03-21T12:49:27.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:27 smithi155 ceph-mon[33846]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-21T12:49:27.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:27 smithi155 ceph-mon[33846]: Deploying daemon osd.3 on smithi043 2024-03-21T12:49:29.435 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:29 smithi043 ceph-mon[30062]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-21T12:49:29.435 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:29 smithi043 ceph-mon[41699]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-21T12:49:29.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:29 smithi155 ceph-mon[33846]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-21T12:49:31.544 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:31 smithi043 ceph-mon[30062]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 74 KiB/s wr, 5 op/s 2024-03-21T12:49:31.545 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:31 smithi043 ceph-mon[41699]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 74 KiB/s wr, 5 op/s 2024-03-21T12:49:31.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:31 smithi155 ceph-mon[33846]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 74 KiB/s wr, 5 op/s 2024-03-21T12:49:33.208 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:33 smithi043 ceph-mon[30062]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-21T12:49:33.209 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:33 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:33.209 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:33 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:33.209 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:33 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:49:33.209 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:33 smithi043 ceph-mon[41699]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-21T12:49:33.209 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:33 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:33.209 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:33 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:33.209 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:33 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:49:33.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:33 smithi155 ceph-mon[33846]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-21T12:49:33.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:33 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:33.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:33 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:33.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:33 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:49:35.228 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:35 smithi043 ceph-mon[41699]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-21T12:49:35.228 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:35 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:35.228 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:35 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:35.228 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:35 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:35.228 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:35 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:49:35.228 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:35 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:35.228 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:35 smithi043 ceph-mon[30062]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-21T12:49:35.229 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:35 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:35.229 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:35 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:35.229 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:35 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:35.229 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:35 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:49:35.229 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:35 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:35.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:35 smithi155 ceph-mon[33846]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-21T12:49:35.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:35 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:35.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:35 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:35.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:35 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:35.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:35 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:49:35.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:35 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:35.998 INFO:teuthology.orchestra.run.smithi043.stdout:Created osd(s) 3 on host 'smithi043' 2024-03-21T12:49:35.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:35.997+0000 7f0f53fff700 1 -- 172.21.15.43:0/3733671395 <== mgr.14152 v1:172.21.15.43:6800/4081968544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 1772299565) 0x7f0f3c000bc0 con 0x7f0f4806f020 2024-03-21T12:49:36.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:35.999+0000 7f0f6100f700 1 -- 172.21.15.43:0/3733671395 >> v1:172.21.15.43:6800/4081968544 conn(0x7f0f4806f020 legacy=0x7f0f480714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:49:36.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:35.999+0000 7f0f6100f700 1 -- 172.21.15.43:0/3733671395 >> v1:172.21.15.43:6790/0 conn(0x7f0f5c108f30 legacy=0x7f0f5c1a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:49:36.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:35.999+0000 7f0f6100f700 1 -- 172.21.15.43:0/3733671395 shutdown_connections 2024-03-21T12:49:36.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:35.999+0000 7f0f6100f700 1 -- 172.21.15.43:0/3733671395 >> 172.21.15.43:0/3733671395 conn(0x7f0f5c0fc420 msgr2=0x7f0f5c10b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:49:36.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:35.999+0000 7f0f6100f700 1 -- 172.21.15.43:0/3733671395 shutdown_connections 2024-03-21T12:49:36.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:49:36.000+0000 7f0f6100f700 1 -- 172.21.15.43:0/3733671395 wait complete. 2024-03-21T12:49:36.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:36 smithi043 ceph-mon[30062]: from='osd.3 v1:172.21.15.43:6813/2246303296' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T12:49:36.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:36 smithi043 ceph-mon[30062]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T12:49:36.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:36 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:36.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:36 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:36.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:36 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:49:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:36 smithi043 ceph-mon[41699]: from='osd.3 v1:172.21.15.43:6813/2246303296' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T12:49:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:36 smithi043 ceph-mon[41699]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T12:49:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:36 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:36 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:36 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:49:36.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:36 smithi155 ceph-mon[33846]: from='osd.3 v1:172.21.15.43:6813/2246303296' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T12:49:36.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:36 smithi155 ceph-mon[33846]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T12:49:36.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:36 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:36.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:36 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:36.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:36 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:49:36.668 DEBUG:teuthology.orchestra.run.smithi043:osd.3> sudo journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@osd.3.service 2024-03-21T12:49:36.670 INFO:tasks.cephadm:Deploying osd.4 on smithi155 with /dev/vg_nvme/lv_4... 2024-03-21T12:49:36.670 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-21T12:49:36.912 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T12:49:37.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:37 smithi043 ceph-mon[41699]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-21T12:49:37.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:37 smithi043 ceph-mon[41699]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-21T12:49:37.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:37 smithi043 ceph-mon[41699]: osdmap e24: 4 total, 3 up, 4 in 2024-03-21T12:49:37.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T12:49:37.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:37 smithi043 ceph-mon[41699]: from='osd.3 v1:172.21.15.43:6813/2246303296' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-21T12:49:37.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:37 smithi043 ceph-mon[41699]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-21T12:49:37.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:37 smithi043 ceph-mon[30062]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-21T12:49:37.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:37 smithi043 ceph-mon[30062]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-21T12:49:37.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:37 smithi043 ceph-mon[30062]: osdmap e24: 4 total, 3 up, 4 in 2024-03-21T12:49:37.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:37 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T12:49:37.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:37 smithi043 ceph-mon[30062]: from='osd.3 v1:172.21.15.43:6813/2246303296' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-21T12:49:37.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:37 smithi043 ceph-mon[30062]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-21T12:49:37.235 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 21 12:49:37 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-3[64567]: 2024-03-21T12:49:37.148+0000 7f76808b8700 -1 osd.3 0 waiting for initial osdmap 2024-03-21T12:49:37.235 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 21 12:49:37 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-3[64567]: 2024-03-21T12:49:37.151+0000 7f767c6ab700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T12:49:37.335 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:37 smithi155 ceph-mon[33846]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-21T12:49:37.335 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:37 smithi155 ceph-mon[33846]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-21T12:49:37.335 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:37 smithi155 ceph-mon[33846]: osdmap e24: 4 total, 3 up, 4 in 2024-03-21T12:49:37.335 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:37 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T12:49:37.335 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:37 smithi155 ceph-mon[33846]: from='osd.3 v1:172.21.15.43:6813/2246303296' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-21T12:49:37.335 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:37 smithi155 ceph-mon[33846]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]: dispatch 2024-03-21T12:49:38.346 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:38 smithi155 ceph-mon[33846]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-03-21T12:49:38.346 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:38 smithi155 ceph-mon[33846]: osdmap e25: 4 total, 3 up, 4 in 2024-03-21T12:49:38.346 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:38 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T12:49:38.347 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:38 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T12:49:38.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:38 smithi043 ceph-mon[30062]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-03-21T12:49:38.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:38 smithi043 ceph-mon[30062]: osdmap e25: 4 total, 3 up, 4 in 2024-03-21T12:49:38.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:38 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T12:49:38.405 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:38 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T12:49:38.405 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:38 smithi043 ceph-mon[41699]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi043", "root=default"]}]': finished 2024-03-21T12:49:38.405 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:38 smithi043 ceph-mon[41699]: osdmap e25: 4 total, 3 up, 4 in 2024-03-21T12:49:38.406 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:38 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T12:49:38.406 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:38 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T12:49:38.899 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-21T12:49:38.900 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-21T12:49:38.900 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-03-21T12:49:38.900 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-03-21T12:49:38.900 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011306 s, 927 MB/s 2024-03-21T12:49:38.900 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-03-21T12:49:39.374 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:39 smithi043 ceph-mon[30062]: purged_snaps scrub starts 2024-03-21T12:49:39.374 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:39 smithi043 ceph-mon[30062]: purged_snaps scrub ok 2024-03-21T12:49:39.374 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:39 smithi043 ceph-mon[30062]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-21T12:49:39.374 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:39 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T12:49:39.374 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:39 smithi043 ceph-mon[30062]: osd.3 v1:172.21.15.43:6813/2246303296 boot 2024-03-21T12:49:39.375 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:39 smithi043 ceph-mon[30062]: osdmap e26: 4 total, 4 up, 4 in 2024-03-21T12:49:39.375 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:39 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T12:49:39.375 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:39 smithi043 ceph-mon[41699]: purged_snaps scrub starts 2024-03-21T12:49:39.375 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:39 smithi043 ceph-mon[41699]: purged_snaps scrub ok 2024-03-21T12:49:39.375 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:39 smithi043 ceph-mon[41699]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-21T12:49:39.375 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:39 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T12:49:39.375 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:39 smithi043 ceph-mon[41699]: osd.3 v1:172.21.15.43:6813/2246303296 boot 2024-03-21T12:49:39.375 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:39 smithi043 ceph-mon[41699]: osdmap e26: 4 total, 4 up, 4 in 2024-03-21T12:49:39.375 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:39 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T12:49:39.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:39 smithi155 ceph-mon[33846]: purged_snaps scrub starts 2024-03-21T12:49:39.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:39 smithi155 ceph-mon[33846]: purged_snaps scrub ok 2024-03-21T12:49:39.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:39 smithi155 ceph-mon[33846]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-21T12:49:39.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:39 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T12:49:39.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:39 smithi155 ceph-mon[33846]: osd.3 v1:172.21.15.43:6813/2246303296 boot 2024-03-21T12:49:39.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:39 smithi155 ceph-mon[33846]: osdmap e26: 4 total, 4 up, 4 in 2024-03-21T12:49:39.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:39 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T12:49:39.546 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi155:vg_nvme/lv_4 2024-03-21T12:49:39.780 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T12:49:40.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:40 smithi155 ceph-mon[33846]: osdmap e27: 4 total, 4 up, 4 in 2024-03-21T12:49:40.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:40 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:40.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:40 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:40.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:40 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:49:40.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:40 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:49:40.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:40 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:49:40.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:40 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:49:40.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:40 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:40.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:40 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:40.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:40 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:49:40.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:40 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:40.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:40 smithi043 ceph-mon[30062]: osdmap e27: 4 total, 4 up, 4 in 2024-03-21T12:49:40.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:40 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:40.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:40 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:40.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:40 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:49:40.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:40 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:49:40.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:40 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:49:40.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:40 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:49:40.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:40 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:40.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:40 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:40.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:40 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:49:40.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:40 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:40.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:40 smithi043 ceph-mon[41699]: osdmap e27: 4 total, 4 up, 4 in 2024-03-21T12:49:40.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:40 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:40.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:40 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:40.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:40 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:49:40.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:40 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:49:40.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:40 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:49:40.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:40 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:49:40.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:40 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:40.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:40 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:40.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:40 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:49:40.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:40 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:41.269 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:41 smithi155 ceph-mon[33846]: Detected new or changed devices on smithi043 2024-03-21T12:49:41.269 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:41 smithi155 ceph-mon[33846]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:41.269 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:41 smithi155 ceph-mon[33846]: Adjusting osd_memory_target on smithi043 to 4017M 2024-03-21T12:49:41.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:41 smithi043 ceph-mon[30062]: Detected new or changed devices on smithi043 2024-03-21T12:49:41.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:41 smithi043 ceph-mon[30062]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:41.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:41 smithi043 ceph-mon[30062]: Adjusting osd_memory_target on smithi043 to 4017M 2024-03-21T12:49:41.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:41 smithi043 ceph-mon[41699]: Detected new or changed devices on smithi043 2024-03-21T12:49:41.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:41 smithi043 ceph-mon[41699]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:41.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:41 smithi043 ceph-mon[41699]: Adjusting osd_memory_target on smithi043 to 4017M 2024-03-21T12:49:41.524 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.523+0000 7fc00812c700 1 -- 172.21.15.155:0/164216461 >> v1:172.21.15.155:6789/0 conn(0x7fc000102ac0 legacy=0x7fc000104eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:49:41.524 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.523+0000 7fc00812c700 1 -- 172.21.15.155:0/164216461 shutdown_connections 2024-03-21T12:49:41.524 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.523+0000 7fc00812c700 1 -- 172.21.15.155:0/164216461 >> 172.21.15.155:0/164216461 conn(0x7fc0000fc450 msgr2=0x7fc0000fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:49:41.525 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.523+0000 7fc00812c700 1 -- 172.21.15.155:0/164216461 shutdown_connections 2024-03-21T12:49:41.525 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.523+0000 7fc00812c700 1 -- 172.21.15.155:0/164216461 wait complete. 2024-03-21T12:49:41.525 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.524+0000 7fc00812c700 1 Processor -- start 2024-03-21T12:49:41.525 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.524+0000 7fc00812c700 1 -- start start 2024-03-21T12:49:41.527 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.524+0000 7fc00812c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc000198240 con 0x7fc00010a0e0 2024-03-21T12:49:41.527 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.524+0000 7fc00812c700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc0001a3a30 con 0x7fc0001065d0 2024-03-21T12:49:41.527 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.524+0000 7fc00812c700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc0001a4c10 con 0x7fc000102ac0 2024-03-21T12:49:41.527 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.524+0000 7fc0056c7700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fc0001065d0 0x7fc00019dbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:39244/0 (socket says 172.21.15.155:39244) 2024-03-21T12:49:41.527 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.524+0000 7fc0056c7700 1 -- 172.21.15.155:0/2888706101 learned_addr learned my addr 172.21.15.155:0/2888706101 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-03-21T12:49:41.527 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.524+0000 7fbff2ffd700 1 -- 172.21.15.155:0/2888706101 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3888089201 0 0) 0x7fc0001a3a30 con 0x7fc0001065d0 2024-03-21T12:49:41.527 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.524+0000 7fbff2ffd700 1 -- 172.21.15.155:0/2888706101 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfec0035b0 con 0x7fc0001065d0 2024-03-21T12:49:41.527 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.525+0000 7fbff2ffd700 1 -- 172.21.15.155:0/2888706101 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1965427045 0 0) 0x7fc0001a4c10 con 0x7fc000102ac0 2024-03-21T12:49:41.527 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.525+0000 7fbff2ffd700 1 -- 172.21.15.155:0/2888706101 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0001a3a30 con 0x7fc000102ac0 2024-03-21T12:49:41.527 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.525+0000 7fbff2ffd700 1 -- 172.21.15.155:0/2888706101 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 846334393 0 0) 0x7fc000198240 con 0x7fc00010a0e0 2024-03-21T12:49:41.527 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.525+0000 7fbff2ffd700 1 -- 172.21.15.155:0/2888706101 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0001a4c10 con 0x7fc00010a0e0 2024-03-21T12:49:41.527 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.525+0000 7fbff2ffd700 1 -- 172.21.15.155:0/2888706101 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3671452722 0 0) 0x7fbfec0035b0 con 0x7fc0001065d0 2024-03-21T12:49:41.528 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.525+0000 7fbff2ffd700 1 -- 172.21.15.155:0/2888706101 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc000198240 con 0x7fc0001065d0 2024-03-21T12:49:41.528 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.525+0000 7fbff2ffd700 1 -- 172.21.15.155:0/2888706101 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fbff40018b0 con 0x7fc0001065d0 2024-03-21T12:49:41.528 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.525+0000 7fbff2ffd700 1 -- 172.21.15.155:0/2888706101 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4107454267 0 0) 0x7fc000198240 con 0x7fc0001065d0 2024-03-21T12:49:41.528 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.525+0000 7fbff2ffd700 1 -- 172.21.15.155:0/2888706101 >> v1:172.21.15.43:6790/0 conn(0x7fc000102ac0 legacy=0x7fc0001976f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:49:41.528 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.525+0000 7fbff2ffd700 1 -- 172.21.15.155:0/2888706101 >> v1:172.21.15.43:6789/0 conn(0x7fc00010a0e0 legacy=0x7fc0001a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:49:41.528 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.525+0000 7fbff2ffd700 1 -- 172.21.15.155:0/2888706101 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0001a5e30 con 0x7fc0001065d0 2024-03-21T12:49:41.528 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.525+0000 7fc00812c700 1 -- 172.21.15.155:0/2888706101 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0001a4e40 con 0x7fc0001065d0 2024-03-21T12:49:41.528 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.525+0000 7fc00812c700 1 -- 172.21.15.155:0/2888706101 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0001a5380 con 0x7fc0001065d0 2024-03-21T12:49:41.528 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.525+0000 7fbff2ffd700 1 -- 172.21.15.155:0/2888706101 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7fbff4003b30 con 0x7fc0001065d0 2024-03-21T12:49:41.528 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.525+0000 7fbff2ffd700 1 -- 172.21.15.155:0/2888706101 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fbff4005ca0 con 0x7fc0001065d0 2024-03-21T12:49:41.528 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.526+0000 7fc00812c700 1 -- 172.21.15.155:0/2888706101 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfe4005150 con 0x7fc0001065d0 2024-03-21T12:49:41.529 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.527+0000 7fbff2ffd700 1 -- 172.21.15.155:0/2888706101 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 15) v1 ==== 92095+0+0 (unknown 2546926431 0 0) 0x7fbff40036a0 con 0x7fc0001065d0 2024-03-21T12:49:41.529 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.527+0000 7fbff2ffd700 1 -- 172.21.15.155:0/2888706101 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(27..27 src has 1..27) v4 ==== 2803+0+0 (unknown 32918239 0 0) 0x7fbff40889e0 con 0x7fc0001065d0 2024-03-21T12:49:41.532 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.532+0000 7fbff2ffd700 1 -- 172.21.15.155:0/2888706101 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7fbff4056e70 con 0x7fc0001065d0 2024-03-21T12:49:41.736 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:41.735+0000 7fc00812c700 1 -- 172.21.15.155:0/2888706101 --> v1:172.21.15.43:6800/4081968544 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fbfe4000bc0 con 0x7fbfec06f070 2024-03-21T12:49:42.446 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:42 smithi155 ceph-mon[33846]: from='client.24223 v1:172.21.15.155:0/2888706101' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:49:42.446 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:42 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T12:49:42.446 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:42 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T12:49:42.446 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:42 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:42.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:42 smithi043 ceph-mon[30062]: from='client.24223 v1:172.21.15.155:0/2888706101' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:49:42.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:42 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T12:49:42.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:42 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T12:49:42.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:42 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:42.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:42 smithi043 ceph-mon[41699]: from='client.24223 v1:172.21.15.155:0/2888706101' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:49:42.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:42 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T12:49:42.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:42 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T12:49:42.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:42 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:43.200 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:43 smithi155 ceph-mon[33846]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:43.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:43 smithi043 ceph-mon[30062]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:43.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:43 smithi043 ceph-mon[41699]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:45.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:45 smithi155 ceph-mon[33846]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:45.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:45 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.155:0/1156445197' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28d67123-5c3e-4973-b1dc-848eb829c590"}]: dispatch 2024-03-21T12:49:45.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:45 smithi155 ceph-mon[33846]: from='client.24229 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28d67123-5c3e-4973-b1dc-848eb829c590"}]: dispatch 2024-03-21T12:49:45.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:45 smithi155 ceph-mon[33846]: from='client.24229 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28d67123-5c3e-4973-b1dc-848eb829c590"}]': finished 2024-03-21T12:49:45.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:45 smithi155 ceph-mon[33846]: osdmap e28: 5 total, 4 up, 5 in 2024-03-21T12:49:45.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:45 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T12:49:45.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:45 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.155:0/3462552474' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T12:49:45.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:45 smithi043 ceph-mon[30062]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:45.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.155:0/1156445197' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28d67123-5c3e-4973-b1dc-848eb829c590"}]: dispatch 2024-03-21T12:49:45.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:45 smithi043 ceph-mon[30062]: from='client.24229 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28d67123-5c3e-4973-b1dc-848eb829c590"}]: dispatch 2024-03-21T12:49:45.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:45 smithi043 ceph-mon[30062]: from='client.24229 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28d67123-5c3e-4973-b1dc-848eb829c590"}]': finished 2024-03-21T12:49:45.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:45 smithi043 ceph-mon[30062]: osdmap e28: 5 total, 4 up, 5 in 2024-03-21T12:49:45.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:45 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T12:49:45.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.155:0/3462552474' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T12:49:45.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:45 smithi043 ceph-mon[41699]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:45.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:45 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.155:0/1156445197' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28d67123-5c3e-4973-b1dc-848eb829c590"}]: dispatch 2024-03-21T12:49:45.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:45 smithi043 ceph-mon[41699]: from='client.24229 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28d67123-5c3e-4973-b1dc-848eb829c590"}]: dispatch 2024-03-21T12:49:45.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:45 smithi043 ceph-mon[41699]: from='client.24229 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28d67123-5c3e-4973-b1dc-848eb829c590"}]': finished 2024-03-21T12:49:45.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:45 smithi043 ceph-mon[41699]: osdmap e28: 5 total, 4 up, 5 in 2024-03-21T12:49:45.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:45 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T12:49:45.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:45 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.155:0/3462552474' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T12:49:47.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:47 smithi155 ceph-mon[33846]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:47.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:47 smithi043 ceph-mon[30062]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:47.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:47 smithi043 ceph-mon[41699]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:49.444 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:49 smithi043 ceph-mon[30062]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:49.444 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:49 smithi043 ceph-mon[41699]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:49.456 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:49 smithi155 ceph-mon[33846]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:51.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:51 smithi155 ceph-mon[33846]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:51.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:51 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T12:49:51.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:51 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:51.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:51 smithi155 ceph-mon[33846]: Deploying daemon osd.4 on smithi155 2024-03-21T12:49:51.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:51 smithi043 ceph-mon[30062]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:51.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:51 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T12:49:51.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:51 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:51.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:51 smithi043 ceph-mon[30062]: Deploying daemon osd.4 on smithi155 2024-03-21T12:49:51.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:51 smithi043 ceph-mon[41699]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:51.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:51 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T12:49:51.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:51 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:51.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:51 smithi043 ceph-mon[41699]: Deploying daemon osd.4 on smithi155 2024-03-21T12:49:53.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:53 smithi155 ceph-mon[33846]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:53.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:53 smithi043 ceph-mon[30062]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:53.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:53 smithi043 ceph-mon[41699]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:55.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:55 smithi155 ceph-mon[33846]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:55.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:55 smithi043 ceph-mon[30062]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:55.483 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:55 smithi043 ceph-mon[41699]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:57.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:57 smithi155 ceph-mon[33846]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:57.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:57 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:57.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:57 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:49:57.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:57 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:57.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:57 smithi043 ceph-mon[30062]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:57.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:57 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:57.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:57 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:49:57.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:57 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:57.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:57 smithi043 ceph-mon[41699]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:57.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:57 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:57.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:57 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:49:57.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:57 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:59.776 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:59 smithi155 ceph-mon[33846]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:59.776 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:59 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:59.776 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:59 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:59.776 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:59 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:59.776 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:59 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:49:59.776 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:59 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:59.776 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:59 smithi155 ceph-mon[33846]: from='osd.4 v1:172.21.15.155:6800/2961807269' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T12:49:59.776 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:49:59 smithi155 ceph-mon[33846]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T12:49:59.854 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:59.855+0000 7fbff2ffd700 1 -- 172.21.15.155:0/2888706101 <== mgr.14152 v1:172.21.15.43:6800/4081968544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 3435391443) 0x7fbfe4000bc0 con 0x7fbfec06f070 2024-03-21T12:49:59.855 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 4 on host 'smithi155' 2024-03-21T12:49:59.857 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:59.857+0000 7fc00812c700 1 -- 172.21.15.155:0/2888706101 >> v1:172.21.15.43:6800/4081968544 conn(0x7fbfec06f070 legacy=0x7fbfec071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:49:59.857 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:59.857+0000 7fc00812c700 1 -- 172.21.15.155:0/2888706101 >> v1:172.21.15.155:6789/0 conn(0x7fc0001065d0 legacy=0x7fc00019dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:49:59.857 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:59.857+0000 7fc00812c700 1 -- 172.21.15.155:0/2888706101 shutdown_connections 2024-03-21T12:49:59.857 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:59.857+0000 7fc00812c700 1 -- 172.21.15.155:0/2888706101 >> 172.21.15.155:0/2888706101 conn(0x7fc0000fc450 msgr2=0x7fc00010c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:49:59.857 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:59.857+0000 7fc00812c700 1 -- 172.21.15.155:0/2888706101 shutdown_connections 2024-03-21T12:49:59.857 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:49:59.857+0000 7fc00812c700 1 -- 172.21.15.155:0/2888706101 wait complete. 2024-03-21T12:49:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:59 smithi043 ceph-mon[30062]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:59 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:59 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:59 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:59 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:49:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:59 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:59 smithi043 ceph-mon[30062]: from='osd.4 v1:172.21.15.155:6800/2961807269' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T12:49:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:49:59 smithi043 ceph-mon[30062]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T12:49:59.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:59 smithi043 ceph-mon[41699]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:49:59.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:59 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:59.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:59 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:59.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:59 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:49:59.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:59 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:49:59.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:59 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:49:59.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:59 smithi043 ceph-mon[41699]: from='osd.4 v1:172.21.15.155:6800/2961807269' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T12:49:59.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:49:59 smithi043 ceph-mon[41699]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T12:50:00.578 DEBUG:teuthology.orchestra.run.smithi155:osd.4> sudo journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@osd.4.service 2024-03-21T12:50:00.580 INFO:tasks.cephadm:Deploying osd.5 on smithi155 with /dev/vg_nvme/lv_3... 2024-03-21T12:50:00.580 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-21T12:50:01.155 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:00 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:01.155 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:00 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:01.155 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:00 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:50:01.155 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:00 smithi155 ceph-mon[33846]: overall HEALTH_OK 2024-03-21T12:50:01.155 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:00 smithi155 ceph-mon[33846]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-21T12:50:01.155 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:00 smithi155 ceph-mon[33846]: osdmap e29: 5 total, 4 up, 5 in 2024-03-21T12:50:01.156 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:00 smithi155 ceph-mon[33846]: from='osd.4 v1:172.21.15.155:6800/2961807269' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-21T12:50:01.156 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:00 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T12:50:01.156 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:00 smithi155 ceph-mon[33846]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-21T12:50:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:00 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:00 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:00 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:50:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:00 smithi043 ceph-mon[30062]: overall HEALTH_OK 2024-03-21T12:50:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:00 smithi043 ceph-mon[30062]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-21T12:50:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:00 smithi043 ceph-mon[30062]: osdmap e29: 5 total, 4 up, 5 in 2024-03-21T12:50:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:00 smithi043 ceph-mon[30062]: from='osd.4 v1:172.21.15.155:6800/2961807269' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-21T12:50:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:00 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T12:50:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:00 smithi043 ceph-mon[30062]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-21T12:50:01.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:00 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:01.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:00 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:01.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:00 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:50:01.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:00 smithi043 ceph-mon[41699]: overall HEALTH_OK 2024-03-21T12:50:01.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:00 smithi043 ceph-mon[41699]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-21T12:50:01.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:00 smithi043 ceph-mon[41699]: osdmap e29: 5 total, 4 up, 5 in 2024-03-21T12:50:01.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:00 smithi043 ceph-mon[41699]: from='osd.4 v1:172.21.15.155:6800/2961807269' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-21T12:50:01.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:00 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T12:50:01.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:00 smithi043 ceph-mon[41699]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-21T12:50:01.380 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T12:50:01.457 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 12:50:01 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-4[39286]: 2024-03-21T12:50:01.160+0000 7f366e34e700 -1 osd.4 0 waiting for initial osdmap 2024-03-21T12:50:01.457 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 12:50:01 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-4[39286]: 2024-03-21T12:50:01.163+0000 7f366a141700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T12:50:02.094 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:01 smithi155 ceph-mon[33846]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:50:02.094 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:01 smithi155 ceph-mon[33846]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-21T12:50:02.095 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:01 smithi155 ceph-mon[33846]: osdmap e30: 5 total, 4 up, 5 in 2024-03-21T12:50:02.095 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:01 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T12:50:02.095 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:01 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T12:50:02.095 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:01 smithi155 ceph-mon[33846]: osd.4 v1:172.21.15.155:6800/2961807269 boot 2024-03-21T12:50:02.095 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:01 smithi155 ceph-mon[33846]: osdmap e31: 5 total, 5 up, 5 in 2024-03-21T12:50:02.095 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:01 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T12:50:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:01 smithi043 ceph-mon[30062]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:50:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:01 smithi043 ceph-mon[30062]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-21T12:50:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:01 smithi043 ceph-mon[30062]: osdmap e30: 5 total, 4 up, 5 in 2024-03-21T12:50:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:01 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T12:50:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:01 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T12:50:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:01 smithi043 ceph-mon[30062]: osd.4 v1:172.21.15.155:6800/2961807269 boot 2024-03-21T12:50:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:01 smithi043 ceph-mon[30062]: osdmap e31: 5 total, 5 up, 5 in 2024-03-21T12:50:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:01 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T12:50:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:01 smithi043 ceph-mon[41699]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:50:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:01 smithi043 ceph-mon[41699]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-21T12:50:02.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:01 smithi043 ceph-mon[41699]: osdmap e30: 5 total, 4 up, 5 in 2024-03-21T12:50:02.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:01 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T12:50:02.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:01 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T12:50:02.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:01 smithi043 ceph-mon[41699]: osd.4 v1:172.21.15.155:6800/2961807269 boot 2024-03-21T12:50:02.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:01 smithi043 ceph-mon[41699]: osdmap e31: 5 total, 5 up, 5 in 2024-03-21T12:50:02.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:01 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T12:50:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:02 smithi155 ceph-mon[33846]: purged_snaps scrub starts 2024-03-21T12:50:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:02 smithi155 ceph-mon[33846]: purged_snaps scrub ok 2024-03-21T12:50:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:02 smithi155 ceph-mon[33846]: osdmap e32: 5 total, 5 up, 5 in 2024-03-21T12:50:03.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:02 smithi043 ceph-mon[30062]: purged_snaps scrub starts 2024-03-21T12:50:03.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:02 smithi043 ceph-mon[30062]: purged_snaps scrub ok 2024-03-21T12:50:03.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:02 smithi043 ceph-mon[30062]: osdmap e32: 5 total, 5 up, 5 in 2024-03-21T12:50:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:02 smithi043 ceph-mon[41699]: purged_snaps scrub starts 2024-03-21T12:50:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:02 smithi043 ceph-mon[41699]: purged_snaps scrub ok 2024-03-21T12:50:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:02 smithi043 ceph-mon[41699]: osdmap e32: 5 total, 5 up, 5 in 2024-03-21T12:50:04.124 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:03 smithi155 ceph-mon[33846]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:50:04.124 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:03 smithi155 ceph-mon[33846]: osdmap e33: 5 total, 5 up, 5 in 2024-03-21T12:50:04.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:03 smithi043 ceph-mon[30062]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:50:04.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:03 smithi043 ceph-mon[30062]: osdmap e33: 5 total, 5 up, 5 in 2024-03-21T12:50:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:03 smithi043 ceph-mon[41699]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-21T12:50:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:03 smithi043 ceph-mon[41699]: osdmap e33: 5 total, 5 up, 5 in 2024-03-21T12:50:04.373 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-21T12:50:04.373 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-21T12:50:04.373 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-03-21T12:50:04.373 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-03-21T12:50:04.374 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0133249 s, 787 MB/s 2024-03-21T12:50:04.374 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-03-21T12:50:05.055 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi155:vg_nvme/lv_3 2024-03-21T12:50:05.295 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T12:50:05.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:05 smithi155 ceph-mon[33846]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-21T12:50:05.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:05 smithi155 ceph-mon[33846]: Detected new or changed devices on smithi155 2024-03-21T12:50:05.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:05 smithi155 ceph-mon[33846]: osdmap e34: 5 total, 5 up, 5 in 2024-03-21T12:50:05.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:05 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:05.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:05 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:05.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:05 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:50:05.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:05 smithi155 ceph-mon[33846]: Adjusting osd_memory_target on smithi155 to 17088M 2024-03-21T12:50:05.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:05 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:05.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:05 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:05.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:05 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:50:05.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:05 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:05 smithi043 ceph-mon[30062]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-21T12:50:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:05 smithi043 ceph-mon[30062]: Detected new or changed devices on smithi155 2024-03-21T12:50:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:05 smithi043 ceph-mon[30062]: osdmap e34: 5 total, 5 up, 5 in 2024-03-21T12:50:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:05 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:05 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:05 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:50:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:05 smithi043 ceph-mon[30062]: Adjusting osd_memory_target on smithi155 to 17088M 2024-03-21T12:50:05.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:05 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:05.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:05 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:05.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:05 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:50:05.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:05 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:05.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:05 smithi043 ceph-mon[41699]: pgmap v119: 1 pgs: 1 peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-21T12:50:05.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:05 smithi043 ceph-mon[41699]: Detected new or changed devices on smithi155 2024-03-21T12:50:05.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:05 smithi043 ceph-mon[41699]: osdmap e34: 5 total, 5 up, 5 in 2024-03-21T12:50:05.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:05 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:05.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:05 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:05.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:05 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:50:05.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:05 smithi043 ceph-mon[41699]: Adjusting osd_memory_target on smithi155 to 17088M 2024-03-21T12:50:05.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:05 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:05.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:05 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:05.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:05 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:50:05.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:05 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:06.657 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:06 smithi155 ceph-mon[33846]: osdmap e35: 5 total, 5 up, 5 in 2024-03-21T12:50:06.887 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.885+0000 7fa5b7f67700 1 -- 172.21.15.155:0/4012072162 >> v1:172.21.15.155:6789/0 conn(0x7fa5b01052b0 legacy=0x7fa5b01076a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:50:06.888 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.885+0000 7fa5b7f67700 1 -- 172.21.15.155:0/4012072162 shutdown_connections 2024-03-21T12:50:06.888 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.885+0000 7fa5b7f67700 1 -- 172.21.15.155:0/4012072162 >> 172.21.15.155:0/4012072162 conn(0x7fa5b00fc450 msgr2=0x7fa5b00fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:50:06.888 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.885+0000 7fa5b7f67700 1 -- 172.21.15.155:0/4012072162 shutdown_connections 2024-03-21T12:50:06.888 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.885+0000 7fa5b7f67700 1 -- 172.21.15.155:0/4012072162 wait complete. 2024-03-21T12:50:06.888 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.885+0000 7fa5b7f67700 1 Processor -- start 2024-03-21T12:50:06.888 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.885+0000 7fa5b7f67700 1 -- start start 2024-03-21T12:50:06.888 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.886+0000 7fa5b7f67700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa5b019a530 con 0x7fa5b0108f40 2024-03-21T12:50:06.888 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.886+0000 7fa5b7f67700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa5b01a5850 con 0x7fa5b0100e90 2024-03-21T12:50:06.888 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.886+0000 7fa5b7f67700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa5b01a6a30 con 0x7fa5b01052b0 2024-03-21T12:50:06.888 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.886+0000 7fa5b5d03700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fa5b0100e90 0x7fa5b01999c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:36266/0 (socket says 172.21.15.155:36266) 2024-03-21T12:50:06.888 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.886+0000 7fa5b5d03700 1 -- 172.21.15.155:0/2847724383 learned_addr learned my addr 172.21.15.155:0/2847724383 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-03-21T12:50:06.888 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.886+0000 7fa5a6ffd700 1 -- 172.21.15.155:0/2847724383 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4256961659 0 0) 0x7fa5b01a5850 con 0x7fa5b0100e90 2024-03-21T12:50:06.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.886+0000 7fa5a6ffd700 1 -- 172.21.15.155:0/2847724383 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa59c0035b0 con 0x7fa5b0100e90 2024-03-21T12:50:06.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.886+0000 7fa5a6ffd700 1 -- 172.21.15.155:0/2847724383 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2525759046 0 0) 0x7fa5b01a6a30 con 0x7fa5b01052b0 2024-03-21T12:50:06.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.886+0000 7fa5a6ffd700 1 -- 172.21.15.155:0/2847724383 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5b01a5850 con 0x7fa5b01052b0 2024-03-21T12:50:06.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.886+0000 7fa5a6ffd700 1 -- 172.21.15.155:0/2847724383 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1166733181 0 0) 0x7fa5b019a530 con 0x7fa5b0108f40 2024-03-21T12:50:06.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.886+0000 7fa5a6ffd700 1 -- 172.21.15.155:0/2847724383 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5b01a6a30 con 0x7fa5b0108f40 2024-03-21T12:50:06.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.886+0000 7fa5a6ffd700 1 -- 172.21.15.155:0/2847724383 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1810100400 0 0) 0x7fa59c0035b0 con 0x7fa5b0100e90 2024-03-21T12:50:06.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.886+0000 7fa5a6ffd700 1 -- 172.21.15.155:0/2847724383 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa5b019a530 con 0x7fa5b0100e90 2024-03-21T12:50:06.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.886+0000 7fa5a6ffd700 1 -- 172.21.15.155:0/2847724383 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fa5ac001890 con 0x7fa5b0100e90 2024-03-21T12:50:06.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.886+0000 7fa5a6ffd700 1 -- 172.21.15.155:0/2847724383 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3583672883 0 0) 0x7fa5b01a5850 con 0x7fa5b01052b0 2024-03-21T12:50:06.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.886+0000 7fa5a6ffd700 1 -- 172.21.15.155:0/2847724383 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa59c0035b0 con 0x7fa5b01052b0 2024-03-21T12:50:06.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.886+0000 7fa5a6ffd700 1 -- 172.21.15.155:0/2847724383 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1449446899 0 0) 0x7fa5b019a530 con 0x7fa5b0100e90 2024-03-21T12:50:06.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.886+0000 7fa5a6ffd700 1 -- 172.21.15.155:0/2847724383 >> v1:172.21.15.43:6790/0 conn(0x7fa5b01052b0 legacy=0x7fa5b019f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:50:06.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.886+0000 7fa5a6ffd700 1 -- 172.21.15.155:0/2847724383 >> v1:172.21.15.43:6789/0 conn(0x7fa5b0108f40 legacy=0x7fa5b01a3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:50:06.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.886+0000 7fa5a6ffd700 1 -- 172.21.15.155:0/2847724383 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b01a7c50 con 0x7fa5b0100e90 2024-03-21T12:50:06.889 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.887+0000 7fa5b7f67700 1 -- 172.21.15.155:0/2847724383 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b01a48a0 con 0x7fa5b0100e90 2024-03-21T12:50:06.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.887+0000 7fa5b7f67700 1 -- 172.21.15.155:0/2847724383 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b01a4de0 con 0x7fa5b0100e90 2024-03-21T12:50:06.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.887+0000 7fa5a6ffd700 1 -- 172.21.15.155:0/2847724383 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7fa5ac003b70 con 0x7fa5b0100e90 2024-03-21T12:50:06.890 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.887+0000 7fa5a6ffd700 1 -- 172.21.15.155:0/2847724383 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fa5ac005d00 con 0x7fa5b0100e90 2024-03-21T12:50:06.890 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:06 smithi043 ceph-mon[30062]: osdmap e35: 5 total, 5 up, 5 in 2024-03-21T12:50:06.891 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:06 smithi043 ceph-mon[41699]: osdmap e35: 5 total, 5 up, 5 in 2024-03-21T12:50:06.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.887+0000 7fa5b7f67700 1 -- 172.21.15.155:0/2847724383 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa594005150 con 0x7fa5b0100e90 2024-03-21T12:50:06.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.888+0000 7fa5a6ffd700 1 -- 172.21.15.155:0/2847724383 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 15) v1 ==== 92095+0+0 (unknown 2546926431 0 0) 0x7fa5ac0036e0 con 0x7fa5b0100e90 2024-03-21T12:50:06.891 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.889+0000 7fa5a6ffd700 1 -- 172.21.15.155:0/2847724383 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(35..35 src has 1..35) v4 ==== 3205+0+0 (unknown 4051641984 0 0) 0x7fa5ac088b00 con 0x7fa5b0100e90 2024-03-21T12:50:06.894 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:06.893+0000 7fa5a6ffd700 1 -- 172.21.15.155:0/2847724383 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7fa5ac056d20 con 0x7fa5b0100e90 2024-03-21T12:50:07.100 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:07.099+0000 7fa5b7f67700 1 -- 172.21.15.155:0/2847724383 --> v1:172.21.15.43:6800/4081968544 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa594000bc0 con 0x7fa59c06edd0 2024-03-21T12:50:07.803 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:07 smithi155 ceph-mon[33846]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-21T12:50:07.803 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:07 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T12:50:07.803 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:07 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T12:50:07.803 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:07 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:07.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:07 smithi043 ceph-mon[30062]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-21T12:50:07.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:07 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T12:50:07.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:07 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T12:50:07.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:07 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:07.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:07 smithi043 ceph-mon[41699]: pgmap v122: 1 pgs: 1 peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-21T12:50:07.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:07 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T12:50:07.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:07 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T12:50:07.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:07 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:08.660 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:08 smithi155 ceph-mon[33846]: from='client.24250 v1:172.21.15.155:0/2847724383' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:50:08.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:08 smithi043 ceph-mon[30062]: from='client.24250 v1:172.21.15.155:0/2847724383' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:50:08.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:08 smithi043 ceph-mon[41699]: from='client.24250 v1:172.21.15.155:0/2847724383' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:50:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:09 smithi155 ceph-mon[33846]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-03-21T12:50:09.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:09 smithi043 ceph-mon[41699]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-03-21T12:50:09.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:09 smithi043 ceph-mon[30062]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 107 KiB/s, 0 objects/s recovering 2024-03-21T12:50:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.155:0/3501528518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3643dae0-6e18-4261-9d47-b3f7606e59e1"}]: dispatch 2024-03-21T12:50:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:10 smithi155 ceph-mon[33846]: from='client.24256 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3643dae0-6e18-4261-9d47-b3f7606e59e1"}]: dispatch 2024-03-21T12:50:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:10 smithi155 ceph-mon[33846]: from='client.24256 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3643dae0-6e18-4261-9d47-b3f7606e59e1"}]': finished 2024-03-21T12:50:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:10 smithi155 ceph-mon[33846]: osdmap e36: 6 total, 5 up, 6 in 2024-03-21T12:50:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:10 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T12:50:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.155:0/88438513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T12:50:10.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.155:0/3501528518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3643dae0-6e18-4261-9d47-b3f7606e59e1"}]: dispatch 2024-03-21T12:50:10.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:10 smithi043 ceph-mon[30062]: from='client.24256 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3643dae0-6e18-4261-9d47-b3f7606e59e1"}]: dispatch 2024-03-21T12:50:10.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:10 smithi043 ceph-mon[30062]: from='client.24256 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3643dae0-6e18-4261-9d47-b3f7606e59e1"}]': finished 2024-03-21T12:50:10.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:10 smithi043 ceph-mon[30062]: osdmap e36: 6 total, 5 up, 6 in 2024-03-21T12:50:10.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:10 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T12:50:10.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.155:0/88438513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T12:50:10.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.155:0/3501528518' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3643dae0-6e18-4261-9d47-b3f7606e59e1"}]: dispatch 2024-03-21T12:50:10.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:10 smithi043 ceph-mon[41699]: from='client.24256 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3643dae0-6e18-4261-9d47-b3f7606e59e1"}]: dispatch 2024-03-21T12:50:10.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:10 smithi043 ceph-mon[41699]: from='client.24256 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3643dae0-6e18-4261-9d47-b3f7606e59e1"}]': finished 2024-03-21T12:50:10.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:10 smithi043 ceph-mon[41699]: osdmap e36: 6 total, 5 up, 6 in 2024-03-21T12:50:10.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:10 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T12:50:10.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.155:0/88438513' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T12:50:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:11 smithi155 ceph-mon[33846]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T12:50:12.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:11 smithi043 ceph-mon[30062]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T12:50:12.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:11 smithi043 ceph-mon[41699]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T12:50:14.139 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:13 smithi155 ceph-mon[33846]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-21T12:50:14.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:13 smithi043 ceph-mon[30062]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-21T12:50:14.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:13 smithi043 ceph-mon[41699]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-21T12:50:16.114 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:15 smithi155 ceph-mon[33846]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-21T12:50:16.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:15 smithi043 ceph-mon[30062]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-21T12:50:16.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:15 smithi043 ceph-mon[41699]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-21T12:50:17.115 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:16 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T12:50:17.115 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:16 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:17.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:16 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T12:50:17.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:16 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:17.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:16 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T12:50:17.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:16 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:17.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:17 smithi043 ceph-mon[30062]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T12:50:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:17 smithi043 ceph-mon[30062]: Deploying daemon osd.5 on smithi155 2024-03-21T12:50:17.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:17 smithi043 ceph-mon[41699]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T12:50:17.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:17 smithi043 ceph-mon[41699]: Deploying daemon osd.5 on smithi155 2024-03-21T12:50:18.203 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:17 smithi155 ceph-mon[33846]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T12:50:18.203 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:17 smithi155 ceph-mon[33846]: Deploying daemon osd.5 on smithi155 2024-03-21T12:50:19.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:19 smithi155 ceph-mon[33846]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-21T12:50:20.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:19 smithi043 ceph-mon[30062]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-21T12:50:20.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:19 smithi043 ceph-mon[41699]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-21T12:50:22.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:21 smithi155 ceph-mon[33846]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-21T12:50:22.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:21 smithi043 ceph-mon[30062]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-21T12:50:22.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:21 smithi043 ceph-mon[41699]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-21T12:50:23.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:23 smithi155 ceph-mon[33846]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-21T12:50:23.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:23 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:23.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:23 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:23.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:23 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:50:23.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:23 smithi043 ceph-mon[30062]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-21T12:50:23.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:23 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:23.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:23 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:23.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:23 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:50:23.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:23 smithi043 ceph-mon[41699]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-21T12:50:23.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:23 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:23.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:23 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:23.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:23 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:50:25.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:25 smithi155 ceph-mon[33846]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-21T12:50:25.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:25 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:25.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:25 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:25.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:25 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:25.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:25 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:50:25.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:25 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:25.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:25 smithi043 ceph-mon[30062]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-21T12:50:25.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:25 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:25.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:25 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:25.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:25 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:25.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:25 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:50:25.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:25 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:25.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:25 smithi043 ceph-mon[41699]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-21T12:50:25.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:25 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:25.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:25 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:25.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:25 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:25.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:25 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:50:25.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:25 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:25.964 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:25.963+0000 7fa5a6ffd700 1 -- 172.21.15.155:0/2847724383 <== mgr.14152 v1:172.21.15.43:6800/4081968544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 2429945703) 0x7fa594000bc0 con 0x7fa59c06edd0 2024-03-21T12:50:25.964 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 5 on host 'smithi155' 2024-03-21T12:50:25.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:25.965+0000 7fa5b7f67700 1 -- 172.21.15.155:0/2847724383 >> v1:172.21.15.43:6800/4081968544 conn(0x7fa59c06edd0 legacy=0x7fa59c071290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:50:25.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:25.965+0000 7fa5b7f67700 1 -- 172.21.15.155:0/2847724383 >> v1:172.21.15.155:6789/0 conn(0x7fa5b0100e90 legacy=0x7fa5b01999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:50:25.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:25.965+0000 7fa5b7f67700 1 -- 172.21.15.155:0/2847724383 shutdown_connections 2024-03-21T12:50:25.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:25.965+0000 7fa5b7f67700 1 -- 172.21.15.155:0/2847724383 >> 172.21.15.155:0/2847724383 conn(0x7fa5b00fc450 msgr2=0x7fa5b010be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:50:25.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:25.965+0000 7fa5b7f67700 1 -- 172.21.15.155:0/2847724383 shutdown_connections 2024-03-21T12:50:25.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:25.965+0000 7fa5b7f67700 1 -- 172.21.15.155:0/2847724383 wait complete. 2024-03-21T12:50:26.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:26 smithi155 ceph-mon[33846]: from='osd.5 v1:172.21.15.155:6804/1838740519' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T12:50:26.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:26 smithi155 ceph-mon[33846]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T12:50:26.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:26 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:26.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:26 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:26.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:26 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:50:26.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:26 smithi043 ceph-mon[41699]: from='osd.5 v1:172.21.15.155:6804/1838740519' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T12:50:26.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:26 smithi043 ceph-mon[41699]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T12:50:26.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:26 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:26.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:26 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:26.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:26 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:50:26.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:26 smithi043 ceph-mon[30062]: from='osd.5 v1:172.21.15.155:6804/1838740519' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T12:50:26.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:26 smithi043 ceph-mon[30062]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T12:50:26.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:26 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:26.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:26 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:26.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:26 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:50:26.640 DEBUG:teuthology.orchestra.run.smithi155:osd.5> sudo journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@osd.5.service 2024-03-21T12:50:26.642 INFO:tasks.cephadm:Deploying osd.6 on smithi155 with /dev/vg_nvme/lv_2... 2024-03-21T12:50:26.642 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-21T12:50:27.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:27 smithi155 ceph-mon[33846]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-21T12:50:27.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:27 smithi155 ceph-mon[33846]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-21T12:50:27.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:27 smithi155 ceph-mon[33846]: from='osd.5 v1:172.21.15.155:6804/1838740519' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-21T12:50:27.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:27 smithi155 ceph-mon[33846]: osdmap e37: 6 total, 5 up, 6 in 2024-03-21T12:50:27.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:27 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T12:50:27.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:27 smithi155 ceph-mon[33846]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-21T12:50:27.483 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T12:50:27.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:27 smithi043 ceph-mon[30062]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-21T12:50:27.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:27 smithi043 ceph-mon[30062]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-21T12:50:27.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:27 smithi043 ceph-mon[30062]: from='osd.5 v1:172.21.15.155:6804/1838740519' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-21T12:50:27.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:27 smithi043 ceph-mon[30062]: osdmap e37: 6 total, 5 up, 6 in 2024-03-21T12:50:27.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:27 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T12:50:27.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:27 smithi043 ceph-mon[30062]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-21T12:50:27.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:27 smithi043 ceph-mon[41699]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-21T12:50:27.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:27 smithi043 ceph-mon[41699]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-21T12:50:27.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:27 smithi043 ceph-mon[41699]: from='osd.5 v1:172.21.15.155:6804/1838740519' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-21T12:50:27.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:27 smithi043 ceph-mon[41699]: osdmap e37: 6 total, 5 up, 6 in 2024-03-21T12:50:27.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:27 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T12:50:27.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:27 smithi043 ceph-mon[41699]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-21T12:50:27.863 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 12:50:27 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T12:50:27.593+0000 7f43ac343700 -1 osd.5 0 waiting for initial osdmap 2024-03-21T12:50:27.863 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 12:50:27 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T12:50:27.598+0000 7f43a412e700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T12:50:28.411 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:28 smithi155 ceph-mon[33846]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-21T12:50:28.411 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:28 smithi155 ceph-mon[33846]: osdmap e38: 6 total, 5 up, 6 in 2024-03-21T12:50:28.412 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:28 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T12:50:28.412 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:28 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T12:50:28.412 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:28 smithi155 ceph-mon[33846]: from='osd.5 ' entity='osd.5' 2024-03-21T12:50:28.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:28 smithi043 ceph-mon[30062]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-21T12:50:28.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:28 smithi043 ceph-mon[30062]: osdmap e38: 6 total, 5 up, 6 in 2024-03-21T12:50:28.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:28 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T12:50:28.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:28 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T12:50:28.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:28 smithi043 ceph-mon[30062]: from='osd.5 ' entity='osd.5' 2024-03-21T12:50:28.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:28 smithi043 ceph-mon[41699]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-21T12:50:28.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:28 smithi043 ceph-mon[41699]: osdmap e38: 6 total, 5 up, 6 in 2024-03-21T12:50:28.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:28 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T12:50:28.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:28 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T12:50:28.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:28 smithi043 ceph-mon[41699]: from='osd.5 ' entity='osd.5' 2024-03-21T12:50:29.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:29 smithi155 ceph-mon[33846]: purged_snaps scrub starts 2024-03-21T12:50:29.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:29 smithi155 ceph-mon[33846]: purged_snaps scrub ok 2024-03-21T12:50:29.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:29 smithi155 ceph-mon[33846]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-21T12:50:29.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:29 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T12:50:29.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:29 smithi155 ceph-mon[33846]: osd.5 v1:172.21.15.155:6804/1838740519 boot 2024-03-21T12:50:29.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:29 smithi155 ceph-mon[33846]: osdmap e39: 6 total, 6 up, 6 in 2024-03-21T12:50:29.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:29 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T12:50:29.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:29 smithi043 ceph-mon[30062]: purged_snaps scrub starts 2024-03-21T12:50:29.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:29 smithi043 ceph-mon[30062]: purged_snaps scrub ok 2024-03-21T12:50:29.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:29 smithi043 ceph-mon[30062]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-21T12:50:29.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:29 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T12:50:29.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:29 smithi043 ceph-mon[30062]: osd.5 v1:172.21.15.155:6804/1838740519 boot 2024-03-21T12:50:29.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:29 smithi043 ceph-mon[30062]: osdmap e39: 6 total, 6 up, 6 in 2024-03-21T12:50:29.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:29 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T12:50:29.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:29 smithi043 ceph-mon[41699]: purged_snaps scrub starts 2024-03-21T12:50:29.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:29 smithi043 ceph-mon[41699]: purged_snaps scrub ok 2024-03-21T12:50:29.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:29 smithi043 ceph-mon[41699]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-03-21T12:50:29.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:29 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T12:50:29.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:29 smithi043 ceph-mon[41699]: osd.5 v1:172.21.15.155:6804/1838740519 boot 2024-03-21T12:50:29.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:29 smithi043 ceph-mon[41699]: osdmap e39: 6 total, 6 up, 6 in 2024-03-21T12:50:29.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:29 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T12:50:30.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:30 smithi155 ceph-mon[33846]: osdmap e40: 6 total, 6 up, 6 in 2024-03-21T12:50:30.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:30 smithi043 ceph-mon[30062]: osdmap e40: 6 total, 6 up, 6 in 2024-03-21T12:50:30.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:30 smithi043 ceph-mon[41699]: osdmap e40: 6 total, 6 up, 6 in 2024-03-21T12:50:30.504 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-21T12:50:30.504 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-21T12:50:30.504 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-03-21T12:50:30.504 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-03-21T12:50:30.504 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0151176 s, 694 MB/s 2024-03-21T12:50:30.504 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-03-21T12:50:31.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:31 smithi155 ceph-mon[33846]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:31.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:31 smithi155 ceph-mon[33846]: osdmap e41: 6 total, 6 up, 6 in 2024-03-21T12:50:31.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:31 smithi155 ceph-mon[33846]: Detected new or changed devices on smithi155 2024-03-21T12:50:31.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:31 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:31.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:31 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:31.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:31 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:50:31.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:31 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:50:31.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:31 smithi155 ceph-mon[33846]: Adjusting osd_memory_target on smithi155 to 8544M 2024-03-21T12:50:31.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:31 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:31.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:31 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:31.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:31 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:50:31.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:31 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:31.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:31 smithi043 ceph-mon[30062]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:31.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:31 smithi043 ceph-mon[30062]: osdmap e41: 6 total, 6 up, 6 in 2024-03-21T12:50:31.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:31 smithi043 ceph-mon[30062]: Detected new or changed devices on smithi155 2024-03-21T12:50:31.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:31 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:31.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:31 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:31.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:31 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:50:31.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:31 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:50:31.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:31 smithi043 ceph-mon[30062]: Adjusting osd_memory_target on smithi155 to 8544M 2024-03-21T12:50:31.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:31 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:31.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:31 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:31.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:31 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:50:31.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:31 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:31.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:31 smithi043 ceph-mon[41699]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:31.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:31 smithi043 ceph-mon[41699]: osdmap e41: 6 total, 6 up, 6 in 2024-03-21T12:50:31.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:31 smithi043 ceph-mon[41699]: Detected new or changed devices on smithi155 2024-03-21T12:50:31.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:31 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:31.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:31 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:31.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:31 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:50:31.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:31 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:50:31.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:31 smithi043 ceph-mon[41699]: Adjusting osd_memory_target on smithi155 to 8544M 2024-03-21T12:50:31.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:31 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:31.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:31 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:31.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:31 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:50:31.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:31 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:31.549 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi155:vg_nvme/lv_2 2024-03-21T12:50:31.788 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T12:50:32.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:32 smithi155 ceph-mon[33846]: osdmap e42: 6 total, 6 up, 6 in 2024-03-21T12:50:32.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:32 smithi043 ceph-mon[30062]: osdmap e42: 6 total, 6 up, 6 in 2024-03-21T12:50:32.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:32 smithi043 ceph-mon[41699]: osdmap e42: 6 total, 6 up, 6 in 2024-03-21T12:50:33.413 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:33 smithi155 ceph-mon[33846]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:33.414 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:33 smithi155 ceph-mon[33846]: osdmap e43: 6 total, 6 up, 6 in 2024-03-21T12:50:33.414 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:33 smithi155 ceph-mon[33846]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-21T12:50:33.421 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.420+0000 7f748ec8e700 1 -- 172.21.15.155:0/2769722726 >> v1:172.21.15.155:6789/0 conn(0x7f748810a0f0 legacy=0x7f748810c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:50:33.421 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.420+0000 7f748ec8e700 1 -- 172.21.15.155:0/2769722726 shutdown_connections 2024-03-21T12:50:33.421 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.420+0000 7f748ec8e700 1 -- 172.21.15.155:0/2769722726 >> 172.21.15.155:0/2769722726 conn(0x7f74880fd540 msgr2=0x7f74880ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:50:33.421 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.420+0000 7f748ec8e700 1 -- 172.21.15.155:0/2769722726 shutdown_connections 2024-03-21T12:50:33.421 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.420+0000 7f748ec8e700 1 -- 172.21.15.155:0/2769722726 wait complete. 2024-03-21T12:50:33.425 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.421+0000 7f748ec8e700 1 Processor -- start 2024-03-21T12:50:33.425 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.421+0000 7f748ec8e700 1 -- start start 2024-03-21T12:50:33.425 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.421+0000 7f748ec8e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f74881a43c0 con 0x7f7488101f80 2024-03-21T12:50:33.425 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.421+0000 7f748ec8e700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f74881a5600 con 0x7f748810a0f0 2024-03-21T12:50:33.425 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.421+0000 7f748ec8e700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f74881a6840 con 0x7f74881063b0 2024-03-21T12:50:33.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.421+0000 7f748d22b700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f748810a0f0 0x7f74881a2ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:41462/0 (socket says 172.21.15.155:41462) 2024-03-21T12:50:33.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.421+0000 7f748d22b700 1 -- 172.21.15.155:0/3132055928 learned_addr learned my addr 172.21.15.155:0/3132055928 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-03-21T12:50:33.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.421+0000 7f7485ffb700 1 -- 172.21.15.155:0/3132055928 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3982503491 0 0) 0x7f74881a5600 con 0x7f748810a0f0 2024-03-21T12:50:33.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.421+0000 7f7485ffb700 1 -- 172.21.15.155:0/3132055928 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74780035b0 con 0x7f748810a0f0 2024-03-21T12:50:33.426 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.421+0000 7f7485ffb700 1 -- 172.21.15.155:0/3132055928 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2747543384 0 0) 0x7f74881a6840 con 0x7f74881063b0 2024-03-21T12:50:33.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.421+0000 7f7485ffb700 1 -- 172.21.15.155:0/3132055928 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74881a5600 con 0x7f74881063b0 2024-03-21T12:50:33.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.421+0000 7f7485ffb700 1 -- 172.21.15.155:0/3132055928 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 30241828 0 0) 0x7f74881a43c0 con 0x7f7488101f80 2024-03-21T12:50:33.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.421+0000 7f7485ffb700 1 -- 172.21.15.155:0/3132055928 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74881a6840 con 0x7f7488101f80 2024-03-21T12:50:33.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.422+0000 7f7485ffb700 1 -- 172.21.15.155:0/3132055928 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 446259256 0 0) 0x7f74780035b0 con 0x7f748810a0f0 2024-03-21T12:50:33.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.422+0000 7f7485ffb700 1 -- 172.21.15.155:0/3132055928 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f74881a43c0 con 0x7f748810a0f0 2024-03-21T12:50:33.427 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.422+0000 7f7485ffb700 1 -- 172.21.15.155:0/3132055928 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f7480004ac0 con 0x7f748810a0f0 2024-03-21T12:50:33.428 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.422+0000 7f7485ffb700 1 -- 172.21.15.155:0/3132055928 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1636834703 0 0) 0x7f74881a43c0 con 0x7f748810a0f0 2024-03-21T12:50:33.428 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.422+0000 7f7485ffb700 1 -- 172.21.15.155:0/3132055928 >> v1:172.21.15.43:6790/0 conn(0x7f74881063b0 legacy=0x7f748819f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:50:33.428 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.422+0000 7f7485ffb700 1 -- 172.21.15.155:0/3132055928 >> v1:172.21.15.43:6789/0 conn(0x7f7488101f80 legacy=0x7f7488075e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:50:33.428 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.422+0000 7f7485ffb700 1 -- 172.21.15.155:0/3132055928 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74881a7a80 con 0x7f748810a0f0 2024-03-21T12:50:33.428 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.422+0000 7f748ec8e700 1 -- 172.21.15.155:0/3132055928 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74881a6a70 con 0x7f748810a0f0 2024-03-21T12:50:33.428 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.422+0000 7f7485ffb700 1 -- 172.21.15.155:0/3132055928 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f7480001f70 con 0x7f748810a0f0 2024-03-21T12:50:33.429 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.422+0000 7f7485ffb700 1 -- 172.21.15.155:0/3132055928 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f74800056a0 con 0x7f748810a0f0 2024-03-21T12:50:33.429 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.422+0000 7f748ec8e700 1 -- 172.21.15.155:0/3132055928 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f74881a6f30 con 0x7f748810a0f0 2024-03-21T12:50:33.429 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.423+0000 7f748ec8e700 1 -- 172.21.15.155:0/3132055928 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7488068a80 con 0x7f748810a0f0 2024-03-21T12:50:33.429 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.423+0000 7f7485ffb700 1 -- 172.21.15.155:0/3132055928 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 15) v1 ==== 92095+0+0 (unknown 2546926431 0 0) 0x7f7480003cc0 con 0x7f748810a0f0 2024-03-21T12:50:33.429 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.424+0000 7f7485ffb700 1 -- 172.21.15.155:0/3132055928 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(43..43 src has 1..43) v4 ==== 3497+0+0 (unknown 2570044370 0 0) 0x7f74800894e0 con 0x7f748810a0f0 2024-03-21T12:50:33.430 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.428+0000 7f7485ffb700 1 -- 172.21.15.155:0/3132055928 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f74800576c0 con 0x7f748810a0f0 2024-03-21T12:50:33.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:33 smithi043 ceph-mon[30062]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:33.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:33 smithi043 ceph-mon[30062]: osdmap e43: 6 total, 6 up, 6 in 2024-03-21T12:50:33.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:33 smithi043 ceph-mon[30062]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-21T12:50:33.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:33 smithi043 ceph-mon[41699]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:33.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:33 smithi043 ceph-mon[41699]: osdmap e43: 6 total, 6 up, 6 in 2024-03-21T12:50:33.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:33 smithi043 ceph-mon[41699]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-03-21T12:50:33.633 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:33.632+0000 7f748ec8e700 1 -- 172.21.15.155:0/3132055928 --> v1:172.21.15.43:6800/4081968544 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f74881079f0 con 0x7f747806f050 2024-03-21T12:50:34.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:34 smithi155 ceph-mon[33846]: from='client.24277 v1:172.21.15.155:0/3132055928' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:50:34.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:34 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T12:50:34.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:34 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T12:50:34.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:34 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:34.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:34 smithi043 ceph-mon[30062]: from='client.24277 v1:172.21.15.155:0/3132055928' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:50:34.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:34 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T12:50:34.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:34 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T12:50:34.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:34 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:34.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:34 smithi043 ceph-mon[41699]: from='client.24277 v1:172.21.15.155:0/3132055928' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:50:34.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:34 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T12:50:34.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:34 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T12:50:34.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:34 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:35.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:35 smithi155 ceph-mon[33846]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:35.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:35 smithi043 ceph-mon[30062]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:35.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:35 smithi043 ceph-mon[41699]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:36.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:36 smithi155 ceph-mon[33846]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-21T12:50:36.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:36 smithi155 ceph-mon[33846]: Cluster is now healthy 2024-03-21T12:50:36.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:36 smithi043 ceph-mon[30062]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-21T12:50:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:36 smithi043 ceph-mon[30062]: Cluster is now healthy 2024-03-21T12:50:36.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:36 smithi043 ceph-mon[41699]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-03-21T12:50:36.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:36 smithi043 ceph-mon[41699]: Cluster is now healthy 2024-03-21T12:50:37.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:37 smithi155 ceph-mon[33846]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:37.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:37 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.155:0/1117255861' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45f89c89-8fa1-455d-9278-8c7ddf0bc422"}]: dispatch 2024-03-21T12:50:37.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:37 smithi155 ceph-mon[33846]: from='client.24283 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45f89c89-8fa1-455d-9278-8c7ddf0bc422"}]: dispatch 2024-03-21T12:50:37.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:37 smithi155 ceph-mon[33846]: from='client.24283 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45f89c89-8fa1-455d-9278-8c7ddf0bc422"}]': finished 2024-03-21T12:50:37.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:37 smithi155 ceph-mon[33846]: osdmap e44: 7 total, 6 up, 7 in 2024-03-21T12:50:37.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:37 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T12:50:37.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:37 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.155:0/849268935' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T12:50:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:37 smithi043 ceph-mon[30062]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:37 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.155:0/1117255861' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45f89c89-8fa1-455d-9278-8c7ddf0bc422"}]: dispatch 2024-03-21T12:50:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:37 smithi043 ceph-mon[30062]: from='client.24283 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45f89c89-8fa1-455d-9278-8c7ddf0bc422"}]: dispatch 2024-03-21T12:50:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:37 smithi043 ceph-mon[30062]: from='client.24283 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45f89c89-8fa1-455d-9278-8c7ddf0bc422"}]': finished 2024-03-21T12:50:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:37 smithi043 ceph-mon[30062]: osdmap e44: 7 total, 6 up, 7 in 2024-03-21T12:50:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:37 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T12:50:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:37 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.155:0/849268935' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T12:50:37.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:37 smithi043 ceph-mon[41699]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:37.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:37 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.155:0/1117255861' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45f89c89-8fa1-455d-9278-8c7ddf0bc422"}]: dispatch 2024-03-21T12:50:37.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:37 smithi043 ceph-mon[41699]: from='client.24283 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "45f89c89-8fa1-455d-9278-8c7ddf0bc422"}]: dispatch 2024-03-21T12:50:37.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:37 smithi043 ceph-mon[41699]: from='client.24283 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "45f89c89-8fa1-455d-9278-8c7ddf0bc422"}]': finished 2024-03-21T12:50:37.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:37 smithi043 ceph-mon[41699]: osdmap e44: 7 total, 6 up, 7 in 2024-03-21T12:50:37.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T12:50:37.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:37 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.155:0/849268935' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T12:50:39.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:39 smithi155 ceph-mon[33846]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-21T12:50:39.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:39 smithi043 ceph-mon[30062]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-21T12:50:39.733 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:39 smithi043 ceph-mon[41699]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 85 KiB/s, 0 objects/s recovering 2024-03-21T12:50:41.683 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:41 smithi155 ceph-mon[33846]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T12:50:41.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:41 smithi043 ceph-mon[30062]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T12:50:41.733 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:41 smithi043 ceph-mon[41699]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T12:50:43.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:43 smithi155 ceph-mon[33846]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-03-21T12:50:43.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:43 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-21T12:50:43.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:43 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:43 smithi043 ceph-mon[30062]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-03-21T12:50:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:43 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-21T12:50:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:43 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:43.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:43 smithi043 ceph-mon[41699]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-03-21T12:50:43.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:43 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-21T12:50:43.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:43 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:44.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:44 smithi155 ceph-mon[33846]: Deploying daemon osd.6 on smithi155 2024-03-21T12:50:44.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:44 smithi043 ceph-mon[30062]: Deploying daemon osd.6 on smithi155 2024-03-21T12:50:44.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:44 smithi043 ceph-mon[41699]: Deploying daemon osd.6 on smithi155 2024-03-21T12:50:45.590 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:45 smithi155 ceph-mon[33846]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T12:50:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:45 smithi043 ceph-mon[30062]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T12:50:45.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:45 smithi043 ceph-mon[41699]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T12:50:47.694 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:47 smithi155 ceph-mon[33846]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:47.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:47 smithi043 ceph-mon[30062]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:47.733 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:47 smithi043 ceph-mon[41699]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:49.625 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:49 smithi155 ceph-mon[33846]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:49.625 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:49 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:49.625 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:49 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:49.625 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:49 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:50:49.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:49 smithi043 ceph-mon[30062]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:49 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:49 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:49 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:50:49.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:49 smithi043 ceph-mon[41699]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:49.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:49 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:49.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:49 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:49.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:49 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:50:51.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:51 smithi155 ceph-mon[33846]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:51.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:51 smithi043 ceph-mon[30062]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:51.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:51 smithi043 ceph-mon[41699]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:52.752 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 6 on host 'smithi155' 2024-03-21T12:50:52.753 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:52.752+0000 7f7485ffb700 1 -- 172.21.15.155:0/3132055928 <== mgr.14152 v1:172.21.15.43:6800/4081968544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 1961374907) 0x7f74881079f0 con 0x7f747806f050 2024-03-21T12:50:52.754 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:52.754+0000 7f748ec8e700 1 -- 172.21.15.155:0/3132055928 >> v1:172.21.15.43:6800/4081968544 conn(0x7f747806f050 legacy=0x7f7478071510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:50:52.754 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:52.754+0000 7f748ec8e700 1 -- 172.21.15.155:0/3132055928 >> v1:172.21.15.155:6789/0 conn(0x7f748810a0f0 legacy=0x7f74881a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:50:52.754 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:52.754+0000 7f748ec8e700 1 -- 172.21.15.155:0/3132055928 shutdown_connections 2024-03-21T12:50:52.755 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:52.754+0000 7f748ec8e700 1 -- 172.21.15.155:0/3132055928 >> 172.21.15.155:0/3132055928 conn(0x7f74880fd540 msgr2=0x7f7488109760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:50:52.755 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:52.754+0000 7f748ec8e700 1 -- 172.21.15.155:0/3132055928 shutdown_connections 2024-03-21T12:50:52.755 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:50:52.754+0000 7f748ec8e700 1 -- 172.21.15.155:0/3132055928 wait complete. 2024-03-21T12:50:52.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:52 smithi155 ceph-mon[33846]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:52.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:52 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:52.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:52 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:52.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:52 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:52.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:52 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:50:52.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:52 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:52.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:52 smithi155 ceph-mon[33846]: from='osd.6 v1:172.21.15.155:6808/2978803316' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T12:50:52.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:52 smithi155 ceph-mon[33846]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T12:50:52.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:52 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:52.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:52 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:52.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:52 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:50:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:52 smithi043 ceph-mon[30062]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:52 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:52 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:52 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:52 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:50:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:52 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:52 smithi043 ceph-mon[30062]: from='osd.6 v1:172.21.15.155:6808/2978803316' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T12:50:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:52 smithi043 ceph-mon[30062]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T12:50:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:52 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:52 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:53.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:52 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:50:53.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:52 smithi043 ceph-mon[41699]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:53.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:52 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:53.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:52 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:53.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:52 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:53.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:52 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:50:53.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:52 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:53.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:52 smithi043 ceph-mon[41699]: from='osd.6 v1:172.21.15.155:6808/2978803316' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T12:50:53.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:52 smithi043 ceph-mon[41699]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T12:50:53.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:52 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:53.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:52 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:53.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:52 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:50:53.399 DEBUG:teuthology.orchestra.run.smithi155:osd.6> sudo journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@osd.6.service 2024-03-21T12:50:53.401 INFO:tasks.cephadm:Deploying osd.7 on smithi155 with /dev/vg_nvme/lv_1... 2024-03-21T12:50:53.401 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-21T12:50:54.186 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T12:50:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:53 smithi043 ceph-mon[30062]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-21T12:50:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:53 smithi043 ceph-mon[30062]: osdmap e45: 7 total, 6 up, 7 in 2024-03-21T12:50:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:53 smithi043 ceph-mon[30062]: from='osd.6 v1:172.21.15.155:6808/2978803316' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-21T12:50:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:53 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T12:50:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:53 smithi043 ceph-mon[30062]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-21T12:50:54.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:53 smithi043 ceph-mon[41699]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-21T12:50:54.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:53 smithi043 ceph-mon[41699]: osdmap e45: 7 total, 6 up, 7 in 2024-03-21T12:50:54.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:53 smithi043 ceph-mon[41699]: from='osd.6 v1:172.21.15.155:6808/2978803316' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-21T12:50:54.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:53 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T12:50:54.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:53 smithi043 ceph-mon[41699]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-21T12:50:54.308 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:53 smithi155 ceph-mon[33846]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-21T12:50:54.308 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:53 smithi155 ceph-mon[33846]: osdmap e45: 7 total, 6 up, 7 in 2024-03-21T12:50:54.308 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:53 smithi155 ceph-mon[33846]: from='osd.6 v1:172.21.15.155:6808/2978803316' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-21T12:50:54.308 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:53 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T12:50:54.308 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:53 smithi155 ceph-mon[33846]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-21T12:50:54.308 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 12:50:53 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T12:50:53.967+0000 7fe2bacaa700 -1 osd.6 0 waiting for initial osdmap 2024-03-21T12:50:54.308 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 12:50:53 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T12:50:53.972+0000 7fe2b6a9d700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T12:50:55.116 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:54 smithi155 ceph-mon[33846]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:55.116 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:54 smithi155 ceph-mon[33846]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-21T12:50:55.116 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:54 smithi155 ceph-mon[33846]: osdmap e46: 7 total, 6 up, 7 in 2024-03-21T12:50:55.117 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:54 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T12:50:55.117 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:54 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T12:50:55.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:54 smithi043 ceph-mon[30062]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:54 smithi043 ceph-mon[30062]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-21T12:50:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:54 smithi043 ceph-mon[30062]: osdmap e46: 7 total, 6 up, 7 in 2024-03-21T12:50:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:54 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T12:50:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:54 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T12:50:55.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:54 smithi043 ceph-mon[41699]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-03-21T12:50:55.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:54 smithi043 ceph-mon[41699]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-21T12:50:55.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:54 smithi043 ceph-mon[41699]: osdmap e46: 7 total, 6 up, 7 in 2024-03-21T12:50:55.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:54 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T12:50:55.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:54 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T12:50:56.222 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:55 smithi155 ceph-mon[33846]: purged_snaps scrub starts 2024-03-21T12:50:56.222 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:55 smithi155 ceph-mon[33846]: purged_snaps scrub ok 2024-03-21T12:50:56.222 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:55 smithi155 ceph-mon[33846]: osd.6 v1:172.21.15.155:6808/2978803316 boot 2024-03-21T12:50:56.222 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:55 smithi155 ceph-mon[33846]: osdmap e47: 7 total, 7 up, 7 in 2024-03-21T12:50:56.222 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:55 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T12:50:56.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:55 smithi043 ceph-mon[30062]: purged_snaps scrub starts 2024-03-21T12:50:56.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:55 smithi043 ceph-mon[30062]: purged_snaps scrub ok 2024-03-21T12:50:56.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:55 smithi043 ceph-mon[30062]: osd.6 v1:172.21.15.155:6808/2978803316 boot 2024-03-21T12:50:56.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:55 smithi043 ceph-mon[30062]: osdmap e47: 7 total, 7 up, 7 in 2024-03-21T12:50:56.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:55 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T12:50:56.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:55 smithi043 ceph-mon[41699]: purged_snaps scrub starts 2024-03-21T12:50:56.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:55 smithi043 ceph-mon[41699]: purged_snaps scrub ok 2024-03-21T12:50:56.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:55 smithi043 ceph-mon[41699]: osd.6 v1:172.21.15.155:6808/2978803316 boot 2024-03-21T12:50:56.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:55 smithi043 ceph-mon[41699]: osdmap e47: 7 total, 7 up, 7 in 2024-03-21T12:50:56.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:55 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T12:50:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:56 smithi155 ceph-mon[33846]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-21T12:50:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:56 smithi155 ceph-mon[33846]: osdmap e48: 7 total, 7 up, 7 in 2024-03-21T12:50:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:56 smithi155 ceph-mon[33846]: osdmap e49: 7 total, 7 up, 7 in 2024-03-21T12:50:57.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:56 smithi043 ceph-mon[30062]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-21T12:50:57.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:56 smithi043 ceph-mon[30062]: osdmap e48: 7 total, 7 up, 7 in 2024-03-21T12:50:57.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:56 smithi043 ceph-mon[30062]: osdmap e49: 7 total, 7 up, 7 in 2024-03-21T12:50:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:56 smithi043 ceph-mon[41699]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-21T12:50:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:56 smithi043 ceph-mon[41699]: osdmap e48: 7 total, 7 up, 7 in 2024-03-21T12:50:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:56 smithi043 ceph-mon[41699]: osdmap e49: 7 total, 7 up, 7 in 2024-03-21T12:50:57.250 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-21T12:50:57.250 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-21T12:50:57.250 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-03-21T12:50:57.250 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-03-21T12:50:57.251 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0140852 s, 744 MB/s 2024-03-21T12:50:57.251 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-03-21T12:50:58.294 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi155:vg_nvme/lv_1 2024-03-21T12:50:58.533 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T12:50:58.910 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:58 smithi155 ceph-mon[33846]: Detected new or changed devices on smithi155 2024-03-21T12:50:58.911 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:58 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:58.911 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:58 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:58.911 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:58 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:50:58.911 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:58 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:50:58.911 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:58 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:50:58.911 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:58 smithi155 ceph-mon[33846]: Adjusting osd_memory_target on smithi155 to 5696M 2024-03-21T12:50:58.911 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:58 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:58.911 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:58 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:58.911 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:58 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:50:58.911 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:58 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:58.911 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:58 smithi155 ceph-mon[33846]: osdmap e50: 7 total, 7 up, 7 in 2024-03-21T12:50:58.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:58 smithi043 ceph-mon[30062]: Detected new or changed devices on smithi155 2024-03-21T12:50:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:58 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:58 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:58 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:50:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:58 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:50:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:58 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:50:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:58 smithi043 ceph-mon[30062]: Adjusting osd_memory_target on smithi155 to 5696M 2024-03-21T12:50:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:58 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:58 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:58 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:50:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:58 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:58 smithi043 ceph-mon[30062]: osdmap e50: 7 total, 7 up, 7 in 2024-03-21T12:50:58.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:58 smithi043 ceph-mon[41699]: Detected new or changed devices on smithi155 2024-03-21T12:50:58.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:58 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:58.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:58 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:58.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:58 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:50:58.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:58 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:50:58.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:58 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:50:58.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:58 smithi043 ceph-mon[41699]: Adjusting osd_memory_target on smithi155 to 5696M 2024-03-21T12:50:58.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:58 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:58.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:58 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:50:58.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:58 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:50:58.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:58 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:50:58.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:58 smithi043 ceph-mon[41699]: osdmap e50: 7 total, 7 up, 7 in 2024-03-21T12:50:59.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:59 smithi155 ceph-mon[33846]: pgmap v162: 1 pgs: 1 remapped+peering; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-21T12:50:59.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:50:59 smithi155 ceph-mon[33846]: osdmap e51: 7 total, 7 up, 7 in 2024-03-21T12:50:59.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:59 smithi043 ceph-mon[30062]: pgmap v162: 1 pgs: 1 remapped+peering; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-21T12:50:59.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:50:59 smithi043 ceph-mon[30062]: osdmap e51: 7 total, 7 up, 7 in 2024-03-21T12:50:59.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:59 smithi043 ceph-mon[41699]: pgmap v162: 1 pgs: 1 remapped+peering; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-21T12:50:59.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:50:59 smithi043 ceph-mon[41699]: osdmap e51: 7 total, 7 up, 7 in 2024-03-21T12:51:00.083 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.079+0000 7f0dec0d2700 1 -- 172.21.15.155:0/1628365509 >> v1:172.21.15.155:6789/0 conn(0x7f0de4108f40 legacy=0x7f0de410b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:00.083 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.079+0000 7f0dec0d2700 1 -- 172.21.15.155:0/1628365509 shutdown_connections 2024-03-21T12:51:00.083 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.079+0000 7f0dec0d2700 1 -- 172.21.15.155:0/1628365509 >> 172.21.15.155:0/1628365509 conn(0x7f0de40fc430 msgr2=0x7f0de40fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:51:00.083 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.079+0000 7f0dec0d2700 1 -- 172.21.15.155:0/1628365509 shutdown_connections 2024-03-21T12:51:00.083 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.079+0000 7f0dec0d2700 1 -- 172.21.15.155:0/1628365509 wait complete. 2024-03-21T12:51:00.083 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.080+0000 7f0dec0d2700 1 Processor -- start 2024-03-21T12:51:00.083 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.080+0000 7f0dec0d2700 1 -- start start 2024-03-21T12:51:00.083 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.080+0000 7f0dec0d2700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0de41971a0 con 0x7f0de4100e90 2024-03-21T12:51:00.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.080+0000 7f0dec0d2700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0de41a2990 con 0x7f0de41052b0 2024-03-21T12:51:00.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.080+0000 7f0dec0d2700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0de41a3b70 con 0x7f0de4108f40 2024-03-21T12:51:00.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.080+0000 7f0de966d700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f0de41052b0 0x7f0de419cb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:57138/0 (socket says 172.21.15.155:57138) 2024-03-21T12:51:00.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.080+0000 7f0de966d700 1 -- 172.21.15.155:0/1744633578 learned_addr learned my addr 172.21.15.155:0/1744633578 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-03-21T12:51:00.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.080+0000 7f0ddaffd700 1 -- 172.21.15.155:0/1744633578 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2502881755 0 0) 0x7f0de41a2990 con 0x7f0de41052b0 2024-03-21T12:51:00.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.080+0000 7f0ddaffd700 1 -- 172.21.15.155:0/1744633578 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dd00035b0 con 0x7f0de41052b0 2024-03-21T12:51:00.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.080+0000 7f0ddaffd700 1 -- 172.21.15.155:0/1744633578 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 455044206 0 0) 0x7f0de41a3b70 con 0x7f0de4108f40 2024-03-21T12:51:00.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.080+0000 7f0ddaffd700 1 -- 172.21.15.155:0/1744633578 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0de41a2990 con 0x7f0de4108f40 2024-03-21T12:51:00.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.080+0000 7f0ddaffd700 1 -- 172.21.15.155:0/1744633578 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 549330459 0 0) 0x7f0de41971a0 con 0x7f0de4100e90 2024-03-21T12:51:00.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.080+0000 7f0ddaffd700 1 -- 172.21.15.155:0/1744633578 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0de41a3b70 con 0x7f0de4100e90 2024-03-21T12:51:00.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.080+0000 7f0ddaffd700 1 -- 172.21.15.155:0/1744633578 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 300620780 0 0) 0x7f0dd00035b0 con 0x7f0de41052b0 2024-03-21T12:51:00.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.080+0000 7f0ddaffd700 1 -- 172.21.15.155:0/1744633578 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0de41971a0 con 0x7f0de41052b0 2024-03-21T12:51:00.084 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.080+0000 7f0ddaffd700 1 -- 172.21.15.155:0/1744633578 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f0de00018f0 con 0x7f0de41052b0 2024-03-21T12:51:00.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.081+0000 7f0ddaffd700 1 -- 172.21.15.155:0/1744633578 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3144651293 0 0) 0x7f0de41a2990 con 0x7f0de4108f40 2024-03-21T12:51:00.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.081+0000 7f0ddaffd700 1 -- 172.21.15.155:0/1744633578 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0dd00035b0 con 0x7f0de4108f40 2024-03-21T12:51:00.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.081+0000 7f0ddaffd700 1 -- 172.21.15.155:0/1744633578 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f0ddc0017b0 con 0x7f0de4108f40 2024-03-21T12:51:00.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.081+0000 7f0ddaffd700 1 -- 172.21.15.155:0/1744633578 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3850030769 0 0) 0x7f0de41a3b70 con 0x7f0de4100e90 2024-03-21T12:51:00.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.081+0000 7f0ddaffd700 1 -- 172.21.15.155:0/1744633578 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0de41a2990 con 0x7f0de4100e90 2024-03-21T12:51:00.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.081+0000 7f0ddaffd700 1 -- 172.21.15.155:0/1744633578 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1970288156 0 0) 0x7f0de41971a0 con 0x7f0de41052b0 2024-03-21T12:51:00.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.081+0000 7f0ddaffd700 1 -- 172.21.15.155:0/1744633578 >> v1:172.21.15.43:6790/0 conn(0x7f0de4108f40 legacy=0x7f0de41a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:00.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.081+0000 7f0ddaffd700 1 -- 172.21.15.155:0/1744633578 >> v1:172.21.15.43:6789/0 conn(0x7f0de4100e90 legacy=0x7f0de4196650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:00.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.081+0000 7f0ddaffd700 1 -- 172.21.15.155:0/1744633578 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de41a4d90 con 0x7f0de41052b0 2024-03-21T12:51:00.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.081+0000 7f0dec0d2700 1 -- 172.21.15.155:0/1744633578 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de41a3da0 con 0x7f0de41052b0 2024-03-21T12:51:00.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.081+0000 7f0dec0d2700 1 -- 172.21.15.155:0/1744633578 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de41a42e0 con 0x7f0de41052b0 2024-03-21T12:51:00.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.081+0000 7f0ddaffd700 1 -- 172.21.15.155:0/1744633578 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f0de0003b70 con 0x7f0de41052b0 2024-03-21T12:51:00.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.081+0000 7f0ddaffd700 1 -- 172.21.15.155:0/1744633578 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f0de0005cb0 con 0x7f0de41052b0 2024-03-21T12:51:00.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.082+0000 7f0dec0d2700 1 -- 172.21.15.155:0/1744633578 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc8005150 con 0x7f0de41052b0 2024-03-21T12:51:00.085 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.082+0000 7f0ddaffd700 1 -- 172.21.15.155:0/1744633578 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 15) v1 ==== 92095+0+0 (unknown 2546926431 0 0) 0x7f0de00036e0 con 0x7f0de41052b0 2024-03-21T12:51:00.086 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.083+0000 7f0ddaffd700 1 -- 172.21.15.155:0/1744633578 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(51..51 src has 1..51) v4 ==== 3789+0+0 (unknown 533655097 0 0) 0x7f0de0088d60 con 0x7f0de41052b0 2024-03-21T12:51:00.088 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.087+0000 7f0ddaffd700 1 -- 172.21.15.155:0/1744633578 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f0de0005c90 con 0x7f0de41052b0 2024-03-21T12:51:00.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:00.289+0000 7f0dec0d2700 1 -- 172.21.15.155:0/1744633578 --> v1:172.21.15.43:6800/4081968544 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f0dc8000bc0 con 0x7f0dd006f0e0 2024-03-21T12:51:00.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:00 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T12:51:00.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:00 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T12:51:00.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:00 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:00.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:00 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T12:51:00.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:00 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T12:51:00.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:00 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:01.006 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:00 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T12:51:01.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:00 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T12:51:01.007 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:00 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:01.938 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:01 smithi155 ceph-mon[33846]: pgmap v165: 1 pgs: 1 remapped+peering; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-21T12:51:01.938 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:01 smithi155 ceph-mon[33846]: from='client.24304 v1:172.21.15.155:0/1744633578' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:51:01.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:01 smithi043 ceph-mon[30062]: pgmap v165: 1 pgs: 1 remapped+peering; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-21T12:51:01.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:01 smithi043 ceph-mon[30062]: from='client.24304 v1:172.21.15.155:0/1744633578' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:51:01.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:01 smithi043 ceph-mon[41699]: pgmap v165: 1 pgs: 1 remapped+peering; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-21T12:51:01.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:01 smithi043 ceph-mon[41699]: from='client.24304 v1:172.21.15.155:0/1744633578' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:51:03.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:03 smithi155 ceph-mon[33846]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-21T12:51:03.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:03 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.155:0/3179088379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56960e89-0f80-4192-95cb-54cf6da32ca7"}]: dispatch 2024-03-21T12:51:03.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:03 smithi155 ceph-mon[33846]: from='client.24310 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56960e89-0f80-4192-95cb-54cf6da32ca7"}]: dispatch 2024-03-21T12:51:03.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:03 smithi155 ceph-mon[33846]: from='client.24310 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56960e89-0f80-4192-95cb-54cf6da32ca7"}]': finished 2024-03-21T12:51:03.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:03 smithi155 ceph-mon[33846]: osdmap e52: 8 total, 7 up, 8 in 2024-03-21T12:51:03.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:03 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T12:51:03.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:03 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.155:0/1507750261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T12:51:03.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:03 smithi043 ceph-mon[30062]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-21T12:51:03.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:03 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.155:0/3179088379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56960e89-0f80-4192-95cb-54cf6da32ca7"}]: dispatch 2024-03-21T12:51:03.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:03 smithi043 ceph-mon[30062]: from='client.24310 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56960e89-0f80-4192-95cb-54cf6da32ca7"}]: dispatch 2024-03-21T12:51:03.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:03 smithi043 ceph-mon[30062]: from='client.24310 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56960e89-0f80-4192-95cb-54cf6da32ca7"}]': finished 2024-03-21T12:51:03.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:03 smithi043 ceph-mon[30062]: osdmap e52: 8 total, 7 up, 8 in 2024-03-21T12:51:03.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:03 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T12:51:03.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:03 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.155:0/1507750261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T12:51:03.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:03 smithi043 ceph-mon[41699]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-21T12:51:03.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:03 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.155:0/3179088379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56960e89-0f80-4192-95cb-54cf6da32ca7"}]: dispatch 2024-03-21T12:51:03.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:03 smithi043 ceph-mon[41699]: from='client.24310 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56960e89-0f80-4192-95cb-54cf6da32ca7"}]: dispatch 2024-03-21T12:51:03.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:03 smithi043 ceph-mon[41699]: from='client.24310 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56960e89-0f80-4192-95cb-54cf6da32ca7"}]': finished 2024-03-21T12:51:03.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:03 smithi043 ceph-mon[41699]: osdmap e52: 8 total, 7 up, 8 in 2024-03-21T12:51:03.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:03 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T12:51:03.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:03 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.155:0/1507750261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T12:51:05.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:05 smithi043 ceph-mon[30062]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T12:51:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:05 smithi043 ceph-mon[41699]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T12:51:06.138 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:05 smithi155 ceph-mon[33846]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T12:51:07.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:07 smithi043 ceph-mon[30062]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T12:51:07.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:07 smithi043 ceph-mon[41699]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T12:51:08.102 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:07 smithi155 ceph-mon[33846]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T12:51:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:09 smithi155 ceph-mon[33846]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-21T12:51:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:09 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-21T12:51:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:09 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:09 smithi043 ceph-mon[30062]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-21T12:51:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:09 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-21T12:51:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:09 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:09.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:09 smithi043 ceph-mon[41699]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-03-21T12:51:09.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:09 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-21T12:51:09.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:09 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:10.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:10 smithi043 ceph-mon[30062]: Deploying daemon osd.7 on smithi155 2024-03-21T12:51:10.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:10 smithi043 ceph-mon[41699]: Deploying daemon osd.7 on smithi155 2024-03-21T12:51:11.199 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:10 smithi155 ceph-mon[33846]: Deploying daemon osd.7 on smithi155 2024-03-21T12:51:11.857 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:11 smithi155 ceph-mon[33846]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T12:51:11.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:11 smithi043 ceph-mon[30062]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T12:51:11.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:11 smithi043 ceph-mon[41699]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T12:51:13.936 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:13 smithi155 ceph-mon[33846]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-21T12:51:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:13 smithi043 ceph-mon[30062]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-21T12:51:13.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:13 smithi043 ceph-mon[41699]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-21T12:51:15.839 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:15 smithi155 ceph-mon[33846]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-21T12:51:15.840 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:15 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:15.840 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:15 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:15.840 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:15 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:51:15.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:15 smithi043 ceph-mon[30062]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-21T12:51:15.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:15 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:15.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:15 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:15.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:15 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:51:15.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:15 smithi043 ceph-mon[41699]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-21T12:51:15.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:15 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:15.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:15 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:15.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:15 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:51:17.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:17 smithi155 ceph-mon[33846]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-21T12:51:18.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:17 smithi043 ceph-mon[30062]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-21T12:51:18.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:17 smithi043 ceph-mon[41699]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-21T12:51:19.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:19 smithi155 ceph-mon[33846]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-21T12:51:19.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:19 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:19.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:19 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:19.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:19 smithi155 ceph-mon[33846]: from='osd.7 v1:172.21.15.155:6812/226070080' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T12:51:19.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:19 smithi155 ceph-mon[33846]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T12:51:19.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:19 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:19.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:19 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:51:19.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:19 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:19.307 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 7 on host 'smithi155' 2024-03-21T12:51:19.307 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:19.305+0000 7f0ddaffd700 1 -- 172.21.15.155:0/1744633578 <== mgr.14152 v1:172.21.15.43:6800/4081968544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 687725071) 0x7f0dc8000bc0 con 0x7f0dd006f0e0 2024-03-21T12:51:19.309 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:19.308+0000 7f0dec0d2700 1 -- 172.21.15.155:0/1744633578 >> v1:172.21.15.43:6800/4081968544 conn(0x7f0dd006f0e0 legacy=0x7f0dd00715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:19.309 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:19.308+0000 7f0dec0d2700 1 -- 172.21.15.155:0/1744633578 >> v1:172.21.15.155:6789/0 conn(0x7f0de41052b0 legacy=0x7f0de419cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:19.309 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:19.308+0000 7f0dec0d2700 1 -- 172.21.15.155:0/1744633578 shutdown_connections 2024-03-21T12:51:19.309 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:19.308+0000 7f0dec0d2700 1 -- 172.21.15.155:0/1744633578 >> 172.21.15.155:0/1744633578 conn(0x7f0de40fc430 msgr2=0x7f0de410b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:51:19.309 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:19.308+0000 7f0dec0d2700 1 -- 172.21.15.155:0/1744633578 shutdown_connections 2024-03-21T12:51:19.309 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:19.308+0000 7f0dec0d2700 1 -- 172.21.15.155:0/1744633578 wait complete. 2024-03-21T12:51:19.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:19 smithi043 ceph-mon[30062]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-21T12:51:19.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:19 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:19.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:19 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:19.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:19 smithi043 ceph-mon[30062]: from='osd.7 v1:172.21.15.155:6812/226070080' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T12:51:19.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:19 smithi043 ceph-mon[30062]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T12:51:19.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:19 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:19.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:19 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:51:19.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:19 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:19.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:19 smithi043 ceph-mon[41699]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-21T12:51:19.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:19 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:19.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:19 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:19.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:19 smithi043 ceph-mon[41699]: from='osd.7 v1:172.21.15.155:6812/226070080' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T12:51:19.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:19 smithi043 ceph-mon[41699]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T12:51:19.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:19 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:19.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:19 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:51:19.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:19 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:19.962 DEBUG:teuthology.orchestra.run.smithi155:osd.7> sudo journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@osd.7.service 2024-03-21T12:51:19.965 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-03-21T12:51:19.965 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-21T12:51:20.085 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:20 smithi043 ceph-mon[30062]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-21T12:51:20.085 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:20 smithi043 ceph-mon[30062]: osdmap e53: 8 total, 7 up, 8 in 2024-03-21T12:51:20.085 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:20 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T12:51:20.085 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:20 smithi043 ceph-mon[41699]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-21T12:51:20.085 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:20 smithi043 ceph-mon[41699]: osdmap e53: 8 total, 7 up, 8 in 2024-03-21T12:51:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:20 smithi155 ceph-mon[33846]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-21T12:51:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:20 smithi155 ceph-mon[33846]: osdmap e53: 8 total, 7 up, 8 in 2024-03-21T12:51:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:20 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T12:51:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:20 smithi155 ceph-mon[33846]: from='osd.7 v1:172.21.15.155:6812/226070080' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-21T12:51:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:20 smithi155 ceph-mon[33846]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-21T12:51:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:20 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:20.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:20 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:20.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:20 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:51:20.208 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 12:51:20 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T12:51:20.088+0000 7fe823629700 -1 osd.7 0 waiting for initial osdmap 2024-03-21T12:51:20.208 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 12:51:20 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T12:51:20.094+0000 7fe81f41c700 -1 osd.7 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T12:51:20.225 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:51:20.347 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:20 smithi043 ceph-mon[30062]: from='osd.7 v1:172.21.15.155:6812/226070080' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-21T12:51:20.347 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:20 smithi043 ceph-mon[30062]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-21T12:51:20.347 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:20 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:20.347 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:20 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:20.347 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:20 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:51:20.347 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:20 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T12:51:20.347 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:20 smithi043 ceph-mon[41699]: from='osd.7 v1:172.21.15.155:6812/226070080' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-21T12:51:20.347 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:20 smithi043 ceph-mon[41699]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-03-21T12:51:20.347 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:20 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:20.348 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:20 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:20.348 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:20 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:51:21.202 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:21 smithi155 ceph-mon[33846]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-21T12:51:21.202 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:21 smithi155 ceph-mon[33846]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-21T12:51:21.202 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:21 smithi155 ceph-mon[33846]: osdmap e54: 8 total, 7 up, 8 in 2024-03-21T12:51:21.202 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:21 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T12:51:21.202 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:21 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T12:51:21.202 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:21 smithi155 ceph-mon[33846]: osd.7 v1:172.21.15.155:6812/226070080 boot 2024-03-21T12:51:21.202 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:21 smithi155 ceph-mon[33846]: osdmap e55: 8 total, 8 up, 8 in 2024-03-21T12:51:21.202 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:21 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T12:51:21.453 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:21 smithi043 ceph-mon[30062]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-21T12:51:21.453 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:21 smithi043 ceph-mon[30062]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-21T12:51:21.453 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:21 smithi043 ceph-mon[30062]: osdmap e54: 8 total, 7 up, 8 in 2024-03-21T12:51:21.453 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:21 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T12:51:21.453 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:21 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T12:51:21.453 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:21 smithi043 ceph-mon[30062]: osd.7 v1:172.21.15.155:6812/226070080 boot 2024-03-21T12:51:21.453 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:21 smithi043 ceph-mon[30062]: osdmap e55: 8 total, 8 up, 8 in 2024-03-21T12:51:21.453 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:21 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T12:51:21.454 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:21 smithi043 ceph-mon[41699]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 197 MiB used, 626 GiB / 626 GiB avail 2024-03-21T12:51:21.454 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:21 smithi043 ceph-mon[41699]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-03-21T12:51:21.454 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:21 smithi043 ceph-mon[41699]: osdmap e54: 8 total, 7 up, 8 in 2024-03-21T12:51:21.454 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:21 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T12:51:21.454 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:21 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T12:51:21.454 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:21 smithi043 ceph-mon[41699]: osd.7 v1:172.21.15.155:6812/226070080 boot 2024-03-21T12:51:21.454 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:21 smithi043 ceph-mon[41699]: osdmap e55: 8 total, 8 up, 8 in 2024-03-21T12:51:21.454 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:21 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T12:51:21.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:21.999+0000 7fabb8266700 1 -- 172.21.15.43:0/717570495 >> v1:172.21.15.43:6789/0 conn(0x7fabb0100e60 legacy=0x7fabb0101280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:22.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:21.999+0000 7fabb8266700 1 -- 172.21.15.43:0/717570495 shutdown_connections 2024-03-21T12:51:22.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:21.999+0000 7fabb8266700 1 -- 172.21.15.43:0/717570495 >> 172.21.15.43:0/717570495 conn(0x7fabb00fc450 msgr2=0x7fabb00fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:51:22.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:21.999+0000 7fabb8266700 1 -- 172.21.15.43:0/717570495 shutdown_connections 2024-03-21T12:51:22.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:21.999+0000 7fabb8266700 1 -- 172.21.15.43:0/717570495 wait complete. 2024-03-21T12:51:22.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.000+0000 7fabb8266700 1 Processor -- start 2024-03-21T12:51:22.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.000+0000 7fabb8266700 1 -- start start 2024-03-21T12:51:22.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.000+0000 7fabb8266700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fabb006b3d0 con 0x7fabb0108f40 2024-03-21T12:51:22.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.000+0000 7fabb8266700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fabb01a34d0 con 0x7fabb0100e60 2024-03-21T12:51:22.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.000+0000 7fabb8266700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fabb01a46f0 con 0x7fabb0105310 2024-03-21T12:51:22.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.000+0000 7fabb5801700 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7fabb0105310 0x7fabb019d670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:46730/0 (socket says 172.21.15.43:46730) 2024-03-21T12:51:22.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.000+0000 7fabb5801700 1 -- 172.21.15.43:0/1584938811 learned_addr learned my addr 172.21.15.43:0/1584938811 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:51:22.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.001+0000 7faba6ffd700 1 -- 172.21.15.43:0/1584938811 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 460037437 0 0) 0x7fabb006b3d0 con 0x7fabb0108f40 2024-03-21T12:51:22.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.001+0000 7faba6ffd700 1 -- 172.21.15.43:0/1584938811 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab9c005240 con 0x7fabb0108f40 2024-03-21T12:51:22.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.001+0000 7faba6ffd700 1 -- 172.21.15.43:0/1584938811 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3107610862 0 0) 0x7fabb01a46f0 con 0x7fabb0105310 2024-03-21T12:51:22.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.001+0000 7faba6ffd700 1 -- 172.21.15.43:0/1584938811 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabb006b3d0 con 0x7fabb0105310 2024-03-21T12:51:22.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.001+0000 7faba6ffd700 1 -- 172.21.15.43:0/1584938811 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804204708 0 0) 0x7fabb01a34d0 con 0x7fabb0100e60 2024-03-21T12:51:22.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.001+0000 7faba6ffd700 1 -- 172.21.15.43:0/1584938811 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabb01a46f0 con 0x7fabb0100e60 2024-03-21T12:51:22.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.001+0000 7faba6ffd700 1 -- 172.21.15.43:0/1584938811 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4056916814 0 0) 0x7fab9c005240 con 0x7fabb0108f40 2024-03-21T12:51:22.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.001+0000 7faba6ffd700 1 -- 172.21.15.43:0/1584938811 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fabb01a34d0 con 0x7fabb0108f40 2024-03-21T12:51:22.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.001+0000 7faba6ffd700 1 -- 172.21.15.43:0/1584938811 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7faba8001db0 con 0x7fabb0108f40 2024-03-21T12:51:22.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.001+0000 7faba6ffd700 1 -- 172.21.15.43:0/1584938811 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 174229020 0 0) 0x7fabb006b3d0 con 0x7fabb0105310 2024-03-21T12:51:22.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.001+0000 7faba6ffd700 1 -- 172.21.15.43:0/1584938811 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fab9c005240 con 0x7fabb0105310 2024-03-21T12:51:22.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.001+0000 7faba6ffd700 1 -- 172.21.15.43:0/1584938811 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fabac0017b0 con 0x7fabb0105310 2024-03-21T12:51:22.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.001+0000 7faba6ffd700 1 -- 172.21.15.43:0/1584938811 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3809253143 0 0) 0x7fabb01a46f0 con 0x7fabb0100e60 2024-03-21T12:51:22.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.001+0000 7faba6ffd700 1 -- 172.21.15.43:0/1584938811 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fabb006b3d0 con 0x7fabb0100e60 2024-03-21T12:51:22.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.001+0000 7faba6ffd700 1 -- 172.21.15.43:0/1584938811 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7faba0001350 con 0x7fabb0100e60 2024-03-21T12:51:22.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.001+0000 7faba6ffd700 1 -- 172.21.15.43:0/1584938811 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2851792313 0 0) 0x7fabb01a34d0 con 0x7fabb0108f40 2024-03-21T12:51:22.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.001+0000 7faba6ffd700 1 -- 172.21.15.43:0/1584938811 >> v1:172.21.15.43:6790/0 conn(0x7fabb0105310 legacy=0x7fabb019d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:22.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.001+0000 7faba6ffd700 1 -- 172.21.15.43:0/1584938811 >> v1:172.21.15.155:6789/0 conn(0x7fabb0100e60 legacy=0x7fabb0199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:22.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.001+0000 7faba6ffd700 1 -- 172.21.15.43:0/1584938811 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb01a5910 con 0x7fabb0108f40 2024-03-21T12:51:22.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.001+0000 7fabb8266700 1 -- 172.21.15.43:0/1584938811 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb01a3700 con 0x7fabb0108f40 2024-03-21T12:51:22.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.001+0000 7fabb8266700 1 -- 172.21.15.43:0/1584938811 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb01a3b60 con 0x7fabb0108f40 2024-03-21T12:51:22.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.002+0000 7faba6ffd700 1 -- 172.21.15.43:0/1584938811 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7faba8002060 con 0x7fabb0108f40 2024-03-21T12:51:22.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.002+0000 7faba6ffd700 1 -- 172.21.15.43:0/1584938811 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7faba8004e30 con 0x7fabb0108f40 2024-03-21T12:51:22.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.002+0000 7fabb8266700 1 -- 172.21.15.43:0/1584938811 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab94005150 con 0x7fabb0108f40 2024-03-21T12:51:22.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.003+0000 7faba6ffd700 1 -- 172.21.15.43:0/1584938811 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 15) v1 ==== 92095+0+0 (unknown 2546926431 0 0) 0x7faba801b880 con 0x7fabb0108f40 2024-03-21T12:51:22.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.003+0000 7faba6ffd700 1 -- 172.21.15.43:0/1584938811 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(56..56 src has 1..56) v4 ==== 4110+0+0 (unknown 2389009530 0 0) 0x7faba8087ff0 con 0x7fabb0108f40 2024-03-21T12:51:22.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.008+0000 7faba6ffd700 1 -- 172.21.15.43:0/1584938811 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7faba8055e90 con 0x7fabb0108f40 2024-03-21T12:51:22.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.214+0000 7fabb8266700 1 -- 172.21.15.43:0/1584938811 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fab94005440 con 0x7fabb0108f40 2024-03-21T12:51:22.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.214+0000 7faba6ffd700 1 -- 172.21.15.43:0/1584938811 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v56) v1 ==== 74+0+130 (unknown 401554801 0 1776546307) 0x7faba8055790 con 0x7fabb0108f40 2024-03-21T12:51:22.214 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:51:22.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.216+0000 7fabb8266700 1 -- 172.21.15.43:0/1584938811 >> v1:172.21.15.43:6800/4081968544 conn(0x7fab9c06f180 legacy=0x7fab9c071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:22.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.216+0000 7fabb8266700 1 -- 172.21.15.43:0/1584938811 >> v1:172.21.15.43:6789/0 conn(0x7fabb0108f40 legacy=0x7fabb01a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:22.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.216+0000 7fabb8266700 1 -- 172.21.15.43:0/1584938811 shutdown_connections 2024-03-21T12:51:22.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.216+0000 7fabb8266700 1 -- 172.21.15.43:0/1584938811 >> 172.21.15.43:0/1584938811 conn(0x7fabb00fc450 msgr2=0x7fabb010b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:51:22.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.216+0000 7fabb8266700 1 -- 172.21.15.43:0/1584938811 shutdown_connections 2024-03-21T12:51:22.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:22.216+0000 7fabb8266700 1 -- 172.21.15.43:0/1584938811 wait complete. 2024-03-21T12:51:22.357 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:22 smithi155 ceph-mon[33846]: purged_snaps scrub starts 2024-03-21T12:51:22.357 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:22 smithi155 ceph-mon[33846]: purged_snaps scrub ok 2024-03-21T12:51:22.357 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:22 smithi155 ceph-mon[33846]: osdmap e56: 8 total, 8 up, 8 in 2024-03-21T12:51:22.483 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:22 smithi043 ceph-mon[41699]: purged_snaps scrub starts 2024-03-21T12:51:22.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:22 smithi043 ceph-mon[41699]: purged_snaps scrub ok 2024-03-21T12:51:22.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:22 smithi043 ceph-mon[41699]: osdmap e56: 8 total, 8 up, 8 in 2024-03-21T12:51:22.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:22 smithi043 ceph-mon[30062]: purged_snaps scrub starts 2024-03-21T12:51:22.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:22 smithi043 ceph-mon[30062]: purged_snaps scrub ok 2024-03-21T12:51:22.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:22 smithi043 ceph-mon[30062]: osdmap e56: 8 total, 8 up, 8 in 2024-03-21T12:51:22.844 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":56,"num_osds":8,"num_up_osds":8,"osd_up_since":1711025481,"num_in_osds":8,"osd_in_since":1711025462,"num_remapped_pgs":1} 2024-03-21T12:51:22.844 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-21T12:51:23.092 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:51:23.210 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[41699]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:51:23.210 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1584938811' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-21T12:51:23.210 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[41699]: osdmap e57: 8 total, 8 up, 8 in 2024-03-21T12:51:23.210 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:23.210 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:23.210 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:51:23.210 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:51:23.211 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:51:23.211 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:51:23.211 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:23.211 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:23.211 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:51:23.211 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:23.211 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[30062]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:51:23.211 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1584938811' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-21T12:51:23.211 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[30062]: osdmap e57: 8 total, 8 up, 8 in 2024-03-21T12:51:23.211 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:23.211 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:23.211 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:51:23.211 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:51:23.212 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:51:23.212 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:51:23.212 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:23.212 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:23.212 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:51:23.212 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:23 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:23.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:23 smithi155 ceph-mon[33846]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:51:23.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:23 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1584938811' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-21T12:51:23.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:23 smithi155 ceph-mon[33846]: osdmap e57: 8 total, 8 up, 8 in 2024-03-21T12:51:23.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:23 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:23.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:23 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:23.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:23 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:51:23.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:23 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:51:23.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:23 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:51:23.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:23 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:51:23.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:23 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:23.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:23 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:23.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:23 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:51:23.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:23 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:24.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.793+0000 7f0e9dfb8700 1 -- 172.21.15.43:0/1135435007 >> v1:172.21.15.43:6790/0 conn(0x7f0e9810ea00 legacy=0x7f0e98110e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.793+0000 7f0e9dfb8700 1 -- 172.21.15.43:0/1135435007 shutdown_connections 2024-03-21T12:51:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.793+0000 7f0e9dfb8700 1 -- 172.21.15.43:0/1135435007 >> 172.21.15.43:0/1135435007 conn(0x7f0e980fc8f0 msgr2=0x7f0e980fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:51:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.793+0000 7f0e9dfb8700 1 -- 172.21.15.43:0/1135435007 shutdown_connections 2024-03-21T12:51:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.793+0000 7f0e9dfb8700 1 -- 172.21.15.43:0/1135435007 wait complete. 2024-03-21T12:51:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.793+0000 7f0e9dfb8700 1 Processor -- start 2024-03-21T12:51:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.794+0000 7f0e9dfb8700 1 -- start start 2024-03-21T12:51:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.794+0000 7f0e9dfb8700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0e981a9160 con 0x7f0e9810ea00 2024-03-21T12:51:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.794+0000 7f0e9dfb8700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0e981aa380 con 0x7f0e9810aef0 2024-03-21T12:51:24.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.794+0000 7f0e9dfb8700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0e981ab5c0 con 0x7f0e981030e0 2024-03-21T12:51:24.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.794+0000 7f0e977fe700 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7f0e981030e0 0x7f0e9810a600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:46744/0 (socket says 172.21.15.43:46744) 2024-03-21T12:51:24.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.794+0000 7f0e977fe700 1 -- 172.21.15.43:0/13996342 learned_addr learned my addr 172.21.15.43:0/13996342 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:51:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.794+0000 7f0e97fff700 1 --1- 172.21.15.43:0/13996342 >> v1:172.21.15.43:6789/0 conn(0x7f0e9810ea00 0x7f0e981a7940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37036/0 (socket says 172.21.15.43:37036) 2024-03-21T12:51:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.794+0000 7f0e94ff9700 1 -- 172.21.15.43:0/13996342 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3857077407 0 0) 0x7f0e981ab5c0 con 0x7f0e981030e0 2024-03-21T12:51:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.794+0000 7f0e94ff9700 1 -- 172.21.15.43:0/13996342 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e800031f0 con 0x7f0e981030e0 2024-03-21T12:51:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.794+0000 7f0e94ff9700 1 -- 172.21.15.43:0/13996342 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1062491871 0 0) 0x7f0e981a9160 con 0x7f0e9810ea00 2024-03-21T12:51:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.794+0000 7f0e94ff9700 1 -- 172.21.15.43:0/13996342 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e981ab5c0 con 0x7f0e9810ea00 2024-03-21T12:51:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.794+0000 7f0e94ff9700 1 -- 172.21.15.43:0/13996342 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1477553939 0 0) 0x7f0e981ab5c0 con 0x7f0e9810ea00 2024-03-21T12:51:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.794+0000 7f0e94ff9700 1 -- 172.21.15.43:0/13996342 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0e981a9160 con 0x7f0e9810ea00 2024-03-21T12:51:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.794+0000 7f0e94ff9700 1 -- 172.21.15.43:0/13996342 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3061022888 0 0) 0x7f0e800031f0 con 0x7f0e981030e0 2024-03-21T12:51:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.795+0000 7f0e94ff9700 1 -- 172.21.15.43:0/13996342 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0e981ab5c0 con 0x7f0e981030e0 2024-03-21T12:51:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.795+0000 7f0e94ff9700 1 -- 172.21.15.43:0/13996342 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3683140636 0 0) 0x7f0e981aa380 con 0x7f0e9810aef0 2024-03-21T12:51:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.795+0000 7f0e94ff9700 1 -- 172.21.15.43:0/13996342 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e800031f0 con 0x7f0e9810aef0 2024-03-21T12:51:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.795+0000 7f0e94ff9700 1 -- 172.21.15.43:0/13996342 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f0e90004720 con 0x7f0e9810ea00 2024-03-21T12:51:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.795+0000 7f0e94ff9700 1 -- 172.21.15.43:0/13996342 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f0e880018b0 con 0x7f0e981030e0 2024-03-21T12:51:24.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.795+0000 7f0e94ff9700 1 -- 172.21.15.43:0/13996342 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3510905753 0 0) 0x7f0e981a9160 con 0x7f0e9810ea00 2024-03-21T12:51:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.795+0000 7f0e94ff9700 1 -- 172.21.15.43:0/13996342 >> v1:172.21.15.43:6790/0 conn(0x7f0e981030e0 legacy=0x7f0e9810a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.795+0000 7f0e94ff9700 1 -- 172.21.15.43:0/13996342 >> v1:172.21.15.155:6789/0 conn(0x7f0e9810aef0 legacy=0x7f0e981a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.795+0000 7f0e94ff9700 1 -- 172.21.15.43:0/13996342 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e981ac800 con 0x7f0e9810ea00 2024-03-21T12:51:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.795+0000 7f0e9dfb8700 1 -- 172.21.15.43:0/13996342 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e981a9390 con 0x7f0e9810ea00 2024-03-21T12:51:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.795+0000 7f0e9dfb8700 1 -- 172.21.15.43:0/13996342 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e981a9850 con 0x7f0e9810ea00 2024-03-21T12:51:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.795+0000 7f0e94ff9700 1 -- 172.21.15.43:0/13996342 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f0e90001dc0 con 0x7f0e9810ea00 2024-03-21T12:51:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.795+0000 7f0e94ff9700 1 -- 172.21.15.43:0/13996342 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f0e900050c0 con 0x7f0e9810ea00 2024-03-21T12:51:24.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.796+0000 7f0e9dfb8700 1 -- 172.21.15.43:0/13996342 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e84005120 con 0x7f0e9810ea00 2024-03-21T12:51:24.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.796+0000 7f0e94ff9700 1 -- 172.21.15.43:0/13996342 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 15) v1 ==== 92095+0+0 (unknown 2546926431 0 0) 0x7f0e90003b70 con 0x7f0e9810ea00 2024-03-21T12:51:24.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.797+0000 7f0e94ff9700 1 -- 172.21.15.43:0/13996342 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(59..59 src has 1..59) v4 ==== 4081+0+0 (unknown 972769531 0 0) 0x7f0e90089390 con 0x7f0e9810ea00 2024-03-21T12:51:24.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:24 smithi043 ceph-mon[30062]: Detected new or changed devices on smithi155 2024-03-21T12:51:24.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:24 smithi043 ceph-mon[30062]: Adjusting osd_memory_target on smithi155 to 4272M 2024-03-21T12:51:24.798 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:24 smithi043 ceph-mon[30062]: osdmap e58: 8 total, 8 up, 8 in 2024-03-21T12:51:24.798 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:24 smithi043 ceph-mon[41699]: Detected new or changed devices on smithi155 2024-03-21T12:51:24.798 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:24 smithi043 ceph-mon[41699]: Adjusting osd_memory_target on smithi155 to 4272M 2024-03-21T12:51:24.798 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:24 smithi043 ceph-mon[41699]: osdmap e58: 8 total, 8 up, 8 in 2024-03-21T12:51:24.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:24.801+0000 7f0e94ff9700 1 -- 172.21.15.43:0/13996342 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f0e900571e0 con 0x7f0e9810ea00 2024-03-21T12:51:24.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:24 smithi155 ceph-mon[33846]: Detected new or changed devices on smithi155 2024-03-21T12:51:24.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:24 smithi155 ceph-mon[33846]: Adjusting osd_memory_target on smithi155 to 4272M 2024-03-21T12:51:24.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:24 smithi155 ceph-mon[33846]: osdmap e58: 8 total, 8 up, 8 in 2024-03-21T12:51:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:25.006+0000 7f0e9dfb8700 1 -- 172.21.15.43:0/13996342 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e84005410 con 0x7f0e9810ea00 2024-03-21T12:51:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:25.007+0000 7f0e94ff9700 1 -- 172.21.15.43:0/13996342 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+11505 (unknown 1651271748 0 2183825769) 0x7f0e90001450 con 0x7f0e9810ea00 2024-03-21T12:51:25.007 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:51:25.008 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":59,"fsid":"c06c8d1a-e780-11ee-95cd-87774f69a715","created":"2024-03-21T12:45:46.690508+0000","modified":"2024-03-21T12:51:24.542376+0000","last_up_change":"2024-03-21T12:51:21.077174+0000","last_in_change":"2024-03-21T12:51:02.931933+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T12:49:12.226146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c030e54-4f13-460e-8ec3-940135cee693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6801","nonce":3260944688}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6802","nonce":3260944688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6804","nonce":3260944688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6803","nonce":3260944688}]},"public_addr":"172.21.15.43:6801/3260944688","cluster_addr":"172.21.15.43:6802/3260944688","heartbeat_back_addr":"172.21.15.43:6804/3260944688","heartbeat_front_addr":"172.21.15.43:6803/3260944688","state":["exists","up"]},{"osd":1,"uuid":"fcc9c3e0-665d-4331-abf3-3ea6e2ac2c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6805","nonce":1862805780}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6806","nonce":1862805780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6808","nonce":1862805780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6807","nonce":1862805780}]},"public_addr":"172.21.15.43:6805/1862805780","cluster_addr":"172.21.15.43:6806/1862805780","heartbeat_back_addr":"172.21.15.43:6808/1862805780","heartbeat_front_addr":"172.21.15.43:6807/1862805780","state":["exists","up"]},{"osd":2,"uuid":"63230fe0-2225-4ebd-a6de-aa55fc7ea0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6809","nonce":1358463812}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6810","nonce":1358463812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6812","nonce":1358463812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6811","nonce":1358463812}]},"public_addr":"172.21.15.43:6809/1358463812","cluster_addr":"172.21.15.43:6810/1358463812","heartbeat_back_addr":"172.21.15.43:6812/1358463812","heartbeat_front_addr":"172.21.15.43:6811/1358463812","state":["exists","up"]},{"osd":3,"uuid":"ac04f1e6-c31c-44b0-909d-a8679d39b335","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6813","nonce":2246303296}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6814","nonce":2246303296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6816","nonce":2246303296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6815","nonce":2246303296}]},"public_addr":"172.21.15.43:6813/2246303296","cluster_addr":"172.21.15.43:6814/2246303296","heartbeat_back_addr":"172.21.15.43:6816/2246303296","heartbeat_front_addr":"172.21.15.43:6815/2246303296","state":["exists","up"]},{"osd":4,"uuid":"28d67123-5c3e-4973-b1dc-848eb829c590","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6800","nonce":2961807269}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6801","nonce":2961807269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6803","nonce":2961807269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6802","nonce":2961807269}]},"public_addr":"172.21.15.155:6800/2961807269","cluster_addr":"172.21.15.155:6801/2961807269","heartbeat_back_addr":"172.21.15.155:6803/2961807269","heartbeat_front_addr":"172.21.15.155:6802/2961807269","state":["exists","up"]},{"osd":5,"uuid":"3643dae0-6e18-4261-9d47-b3f7606e59e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6804","nonce":1838740519}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6805","nonce":1838740519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6807","nonce":1838740519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6806","nonce":1838740519}]},"public_addr":"172.21.15.155:6804/1838740519","cluster_addr":"172.21.15.155:6805/1838740519","heartbeat_back_addr":"172.21.15.155:6807/1838740519","heartbeat_front_addr":"172.21.15.155:6806/1838740519","state":["exists","up"]},{"osd":6,"uuid":"45f89c89-8fa1-455d-9278-8c7ddf0bc422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6808","nonce":2978803316}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6809","nonce":2978803316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6811","nonce":2978803316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6810","nonce":2978803316}]},"public_addr":"172.21.15.155:6808/2978803316","cluster_addr":"172.21.15.155:6809/2978803316","heartbeat_back_addr":"172.21.15.155:6811/2978803316","heartbeat_front_addr":"172.21.15.155:6810/2978803316","state":["exists","up"]},{"osd":7,"uuid":"56960e89-0f80-4192-95cb-54cf6da32ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6812","nonce":226070080}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6813","nonce":226070080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6815","nonce":226070080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6814","nonce":226070080}]},"public_addr":"172.21.15.155:6812/226070080","cluster_addr":"172.21.15.155:6813/226070080","heartbeat_back_addr":"172.21.15.155:6815/226070080","heartbeat_front_addr":"172.21.15.155:6814/226070080","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T12:48:17.270391+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T12:48:43.552605+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T12:49:09.886944+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T12:49:36.259542+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T12:50:00.234269+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T12:50:26.271712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T12:50:53.074703+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T12:51:19.360649+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/2708116611":"2024-03-22T12:46:51.885437+0000","172.21.15.43:6800/2056453291":"2024-03-22T12:46:23.703537+0000","172.21.15.43:6800/1388905519":"2024-03-22T12:46:51.885437+0000","172.21.15.43:0/1502364308":"2024-03-22T12:46:23.703537+0000","172.21.15.43:0/404209980":"2024-03-22T12:46:51.885437+0000","172.21.15.43:0/1693637100":"2024-03-22T12:46:23.703537+0000","172.21.15.43:0/2906079211":"2024-03-22T12:46:23.703537+0000","172.21.15.43:0/1705359135":"2024-03-22T12:46:51.885437+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T12:51:25.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:25.009+0000 7f0e9dfb8700 1 -- 172.21.15.43:0/13996342 >> v1:172.21.15.43:6800/4081968544 conn(0x7f0e8006efa0 legacy=0x7f0e80071460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:25.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:25.009+0000 7f0e9dfb8700 1 -- 172.21.15.43:0/13996342 >> v1:172.21.15.43:6789/0 conn(0x7f0e9810ea00 legacy=0x7f0e981a7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:25.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:25.009+0000 7f0e9dfb8700 1 -- 172.21.15.43:0/13996342 shutdown_connections 2024-03-21T12:51:25.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:25.009+0000 7f0e9dfb8700 1 -- 172.21.15.43:0/13996342 >> 172.21.15.43:0/13996342 conn(0x7f0e980fc8f0 msgr2=0x7f0e98110e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:51:25.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:25.009+0000 7f0e9dfb8700 1 -- 172.21.15.43:0/13996342 shutdown_connections 2024-03-21T12:51:25.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:25.009+0000 7f0e9dfb8700 1 -- 172.21.15.43:0/13996342 wait complete. 2024-03-21T12:51:25.615 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T12:49:12.226146+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T12:51:25.615 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-21T12:51:25.859 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:51:25.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:25 smithi155 ceph-mon[33846]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:51:25.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:25 smithi155 ceph-mon[33846]: osdmap e59: 8 total, 8 up, 8 in 2024-03-21T12:51:25.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:25 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/13996342' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T12:51:25.980 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:25 smithi043 ceph-mon[30062]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:51:25.981 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:25 smithi043 ceph-mon[30062]: osdmap e59: 8 total, 8 up, 8 in 2024-03-21T12:51:25.981 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:25 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/13996342' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T12:51:25.981 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:25 smithi043 ceph-mon[41699]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 224 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:51:25.981 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:25 smithi043 ceph-mon[41699]: osdmap e59: 8 total, 8 up, 8 in 2024-03-21T12:51:25.981 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:25 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/13996342' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T12:51:27.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:27 smithi043 ceph-mon[30062]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:51:27.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:27 smithi043 ceph-mon[41699]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:51:27.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.779+0000 7f8d2fd4c700 1 -- 172.21.15.43:0/1878252928 >> v1:172.21.15.43:6789/0 conn(0x7f8d2810e9d0 legacy=0x7f8d28110e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:27.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.780+0000 7f8d2fd4c700 1 -- 172.21.15.43:0/1878252928 shutdown_connections 2024-03-21T12:51:27.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.780+0000 7f8d2fd4c700 1 -- 172.21.15.43:0/1878252928 >> 172.21.15.43:0/1878252928 conn(0x7f8d280fda70 msgr2=0x7f8d280ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:51:27.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.780+0000 7f8d2fd4c700 1 -- 172.21.15.43:0/1878252928 shutdown_connections 2024-03-21T12:51:27.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.780+0000 7f8d2fd4c700 1 -- 172.21.15.43:0/1878252928 wait complete. 2024-03-21T12:51:27.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.780+0000 7f8d2fd4c700 1 Processor -- start 2024-03-21T12:51:27.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.780+0000 7f8d2fd4c700 1 -- start start 2024-03-21T12:51:27.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.781+0000 7f8d2fd4c700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8d2819ee50 con 0x7f8d2810e9d0 2024-03-21T12:51:27.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.781+0000 7f8d2fd4c700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8d281aea60 con 0x7f8d2810ae30 2024-03-21T12:51:27.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.781+0000 7f8d2fd4c700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8d281afc40 con 0x7f8d281021a0 2024-03-21T12:51:27.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.781+0000 7f8d2e2e9700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8d2810e9d0 0x7f8d2819e520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37046/0 (socket says 172.21.15.43:37046) 2024-03-21T12:51:27.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.781+0000 7f8d2dae8700 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7f8d281021a0 0x7f8d281074c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:46762/0 (socket says 172.21.15.43:46762) 2024-03-21T12:51:27.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.781+0000 7f8d2e2e9700 1 -- 172.21.15.43:0/2176638465 learned_addr learned my addr 172.21.15.43:0/2176638465 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:51:27.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.781+0000 7f8d2d2e7700 1 --1- 172.21.15.43:0/2176638465 >> v1:172.21.15.155:6789/0 conn(0x7f8d2810ae30 0x7f8d2819de10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.43:41528/0 (socket says 172.21.15.43:41528) 2024-03-21T12:51:27.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.781+0000 7f8d1effd700 1 -- 172.21.15.43:0/2176638465 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214822760 0 0) 0x7f8d2819ee50 con 0x7f8d2810e9d0 2024-03-21T12:51:27.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.781+0000 7f8d1effd700 1 -- 172.21.15.43:0/2176638465 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d140031f0 con 0x7f8d2810e9d0 2024-03-21T12:51:27.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.781+0000 7f8d1effd700 1 -- 172.21.15.43:0/2176638465 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 989515697 0 0) 0x7f8d281afc40 con 0x7f8d281021a0 2024-03-21T12:51:27.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.781+0000 7f8d1effd700 1 -- 172.21.15.43:0/2176638465 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d2819ee50 con 0x7f8d281021a0 2024-03-21T12:51:27.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.781+0000 7f8d1effd700 1 -- 172.21.15.43:0/2176638465 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4133655975 0 0) 0x7f8d281aea60 con 0x7f8d2810ae30 2024-03-21T12:51:27.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.781+0000 7f8d1effd700 1 -- 172.21.15.43:0/2176638465 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d281afc40 con 0x7f8d2810ae30 2024-03-21T12:51:27.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.781+0000 7f8d1effd700 1 -- 172.21.15.43:0/2176638465 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1312878843 0 0) 0x7f8d140031f0 con 0x7f8d2810e9d0 2024-03-21T12:51:27.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.781+0000 7f8d1effd700 1 -- 172.21.15.43:0/2176638465 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8d281aea60 con 0x7f8d2810e9d0 2024-03-21T12:51:27.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.781+0000 7f8d1effd700 1 -- 172.21.15.43:0/2176638465 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 364021209 0 0) 0x7f8d2819ee50 con 0x7f8d281021a0 2024-03-21T12:51:27.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.781+0000 7f8d1effd700 1 -- 172.21.15.43:0/2176638465 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8d140031f0 con 0x7f8d281021a0 2024-03-21T12:51:27.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.781+0000 7f8d1effd700 1 -- 172.21.15.43:0/2176638465 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f8d200047b0 con 0x7f8d2810e9d0 2024-03-21T12:51:27.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.781+0000 7f8d1effd700 1 -- 172.21.15.43:0/2176638465 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f8d24001830 con 0x7f8d281021a0 2024-03-21T12:51:27.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.782+0000 7f8d1effd700 1 -- 172.21.15.43:0/2176638465 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 28248830 0 0) 0x7f8d281afc40 con 0x7f8d2810ae30 2024-03-21T12:51:27.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.782+0000 7f8d1effd700 1 -- 172.21.15.43:0/2176638465 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8d2819ee50 con 0x7f8d2810ae30 2024-03-21T12:51:27.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.782+0000 7f8d1effd700 1 -- 172.21.15.43:0/2176638465 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f8d180013d0 con 0x7f8d2810ae30 2024-03-21T12:51:27.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.782+0000 7f8d1effd700 1 -- 172.21.15.43:0/2176638465 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 768303111 0 0) 0x7f8d281aea60 con 0x7f8d2810e9d0 2024-03-21T12:51:27.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.782+0000 7f8d1effd700 1 -- 172.21.15.43:0/2176638465 >> v1:172.21.15.43:6790/0 conn(0x7f8d281021a0 legacy=0x7f8d281074c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:27.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.782+0000 7f8d1effd700 1 -- 172.21.15.43:0/2176638465 >> v1:172.21.15.155:6789/0 conn(0x7f8d2810ae30 legacy=0x7f8d2819de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:27.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.782+0000 7f8d1effd700 1 -- 172.21.15.43:0/2176638465 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d281b0e20 con 0x7f8d2810e9d0 2024-03-21T12:51:27.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.782+0000 7f8d2fd4c700 1 -- 172.21.15.43:0/2176638465 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d281afe70 con 0x7f8d2810e9d0 2024-03-21T12:51:27.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.782+0000 7f8d2fd4c700 1 -- 172.21.15.43:0/2176638465 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d281b0330 con 0x7f8d2810e9d0 2024-03-21T12:51:27.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.782+0000 7f8d1effd700 1 -- 172.21.15.43:0/2176638465 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f8d20001da0 con 0x7f8d2810e9d0 2024-03-21T12:51:27.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.782+0000 7f8d1effd700 1 -- 172.21.15.43:0/2176638465 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f8d20005050 con 0x7f8d2810e9d0 2024-03-21T12:51:27.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.783+0000 7f8d2fd4c700 1 -- 172.21.15.43:0/2176638465 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d0c005120 con 0x7f8d2810e9d0 2024-03-21T12:51:27.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.783+0000 7f8d1effd700 1 -- 172.21.15.43:0/2176638465 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 15) v1 ==== 92095+0+0 (unknown 2546926431 0 0) 0x7f8d200052b0 con 0x7f8d2810e9d0 2024-03-21T12:51:27.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.784+0000 7f8d1effd700 1 -- 172.21.15.43:0/2176638465 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(59..59 src has 1..59) v4 ==== 4081+0+0 (unknown 972769531 0 0) 0x7f8d20089290 con 0x7f8d2810e9d0 2024-03-21T12:51:27.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.788+0000 7f8d1effd700 1 -- 172.21.15.43:0/2176638465 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f8d20057140 con 0x7f8d2810e9d0 2024-03-21T12:51:27.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:27 smithi155 ceph-mon[33846]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:51:27.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.990+0000 7f8d2fd4c700 1 -- 172.21.15.43:0/2176638465 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8d0c005c20 con 0x7f8d2810e9d0 2024-03-21T12:51:27.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.991+0000 7f8d1effd700 1 -- 172.21.15.43:0/2176638465 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (unknown 4013917427 0 2170607528) 0x7f8d20005050 con 0x7f8d2810e9d0 2024-03-21T12:51:27.991 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-03-21T12:51:27.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.993+0000 7f8d2fd4c700 1 -- 172.21.15.43:0/2176638465 >> v1:172.21.15.43:6800/4081968544 conn(0x7f8d1406f0b0 legacy=0x7f8d14071570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:27.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.993+0000 7f8d2fd4c700 1 -- 172.21.15.43:0/2176638465 >> v1:172.21.15.43:6789/0 conn(0x7f8d2810e9d0 legacy=0x7f8d2819e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:27.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.993+0000 7f8d2fd4c700 1 -- 172.21.15.43:0/2176638465 shutdown_connections 2024-03-21T12:51:27.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.993+0000 7f8d2fd4c700 1 -- 172.21.15.43:0/2176638465 >> 172.21.15.43:0/2176638465 conn(0x7f8d280fda70 msgr2=0x7f8d28111b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:51:27.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.993+0000 7f8d2fd4c700 1 -- 172.21.15.43:0/2176638465 shutdown_connections 2024-03-21T12:51:27.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:51:27.993+0000 7f8d2fd4c700 1 -- 172.21.15.43:0/2176638465 wait complete. 2024-03-21T12:51:28.691 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2176638465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T12:51:28.691 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2176638465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T12:51:28.724 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi043 2024-03-21T12:51:28.724 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi043=foo.a' 2024-03-21T12:51:28.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2176638465' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T12:51:28.974 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T12:51:29.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:29 smithi155 ceph-mon[33846]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T12:51:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:29 smithi043 ceph-mon[30062]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T12:51:29.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:29 smithi043 ceph-mon[41699]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T12:51:30.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.598+0000 7f4611f4e700 1 -- 172.21.15.155:0/2896284473 >> v1:172.21.15.155:6789/0 conn(0x7f460c10a130 legacy=0x7f460c10c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:30.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.598+0000 7f4611f4e700 1 -- 172.21.15.155:0/2896284473 shutdown_connections 2024-03-21T12:51:30.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.598+0000 7f4611f4e700 1 -- 172.21.15.155:0/2896284473 >> 172.21.15.155:0/2896284473 conn(0x7f460c0fd560 msgr2=0x7f460c0ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:51:30.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.598+0000 7f4611f4e700 1 -- 172.21.15.155:0/2896284473 shutdown_connections 2024-03-21T12:51:30.600 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.598+0000 7f4611f4e700 1 -- 172.21.15.155:0/2896284473 wait complete. 2024-03-21T12:51:30.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.599+0000 7f4611f4e700 1 Processor -- start 2024-03-21T12:51:30.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.599+0000 7f4611f4e700 1 -- start start 2024-03-21T12:51:30.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.599+0000 7f4611f4e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f460c197c90 con 0x7f460c1063f0 2024-03-21T12:51:30.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.599+0000 7f4611f4e700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f460c1a3480 con 0x7f460c101fc0 2024-03-21T12:51:30.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.599+0000 7f4611f4e700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f460c1a4660 con 0x7f460c10a130 2024-03-21T12:51:30.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.599+0000 7f460b7fe700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f460c101fc0 0x7f460c197140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:35174/0 (socket says 172.21.15.155:35174) 2024-03-21T12:51:30.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.599+0000 7f460b7fe700 1 -- 172.21.15.155:0/215606841 learned_addr learned my addr 172.21.15.155:0/215606841 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-03-21T12:51:30.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.599+0000 7f46097fa700 1 -- 172.21.15.155:0/215606841 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2719638095 0 0) 0x7f460c1a3480 con 0x7f460c101fc0 2024-03-21T12:51:30.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.599+0000 7f46097fa700 1 -- 172.21.15.155:0/215606841 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45f80035b0 con 0x7f460c101fc0 2024-03-21T12:51:30.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.600+0000 7f46097fa700 1 -- 172.21.15.155:0/215606841 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541392407 0 0) 0x7f460c197c90 con 0x7f460c1063f0 2024-03-21T12:51:30.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.600+0000 7f46097fa700 1 -- 172.21.15.155:0/215606841 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f460c1a3480 con 0x7f460c1063f0 2024-03-21T12:51:30.601 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.600+0000 7f46097fa700 1 -- 172.21.15.155:0/215606841 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2288251750 0 0) 0x7f460c1a4660 con 0x7f460c10a130 2024-03-21T12:51:30.602 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.600+0000 7f46097fa700 1 -- 172.21.15.155:0/215606841 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f460c197c90 con 0x7f460c10a130 2024-03-21T12:51:30.602 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.600+0000 7f46097fa700 1 -- 172.21.15.155:0/215606841 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2137107153 0 0) 0x7f45f80035b0 con 0x7f460c101fc0 2024-03-21T12:51:30.602 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.600+0000 7f46097fa700 1 -- 172.21.15.155:0/215606841 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f460c1a4660 con 0x7f460c101fc0 2024-03-21T12:51:30.602 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.600+0000 7f46097fa700 1 -- 172.21.15.155:0/215606841 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f45fc004060 con 0x7f460c101fc0 2024-03-21T12:51:30.602 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.600+0000 7f46097fa700 1 -- 172.21.15.155:0/215606841 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 135466120 0 0) 0x7f460c1a4660 con 0x7f460c101fc0 2024-03-21T12:51:30.602 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.600+0000 7f46097fa700 1 -- 172.21.15.155:0/215606841 >> v1:172.21.15.43:6790/0 conn(0x7f460c10a130 legacy=0x7f460c1a0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:30.602 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.600+0000 7f46097fa700 1 -- 172.21.15.155:0/215606841 >> v1:172.21.15.43:6789/0 conn(0x7f460c1063f0 legacy=0x7f460c19d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:30.602 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.600+0000 7f46097fa700 1 -- 172.21.15.155:0/215606841 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460c1a5880 con 0x7f460c101fc0 2024-03-21T12:51:30.602 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.600+0000 7f4611f4e700 1 -- 172.21.15.155:0/215606841 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f460c1a4890 con 0x7f460c101fc0 2024-03-21T12:51:30.602 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.600+0000 7f4611f4e700 1 -- 172.21.15.155:0/215606841 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f460c1a4d90 con 0x7f460c101fc0 2024-03-21T12:51:30.602 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.600+0000 7f46097fa700 1 -- 172.21.15.155:0/215606841 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f45fc004450 con 0x7f460c101fc0 2024-03-21T12:51:30.602 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.600+0000 7f46097fa700 1 -- 172.21.15.155:0/215606841 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f45fc005d10 con 0x7f460c101fc0 2024-03-21T12:51:30.603 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.601+0000 7f4611f4e700 1 -- 172.21.15.155:0/215606841 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45ec005150 con 0x7f460c101fc0 2024-03-21T12:51:30.603 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.601+0000 7f46097fa700 1 -- 172.21.15.155:0/215606841 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 15) v1 ==== 92095+0+0 (unknown 2546926431 0 0) 0x7f45fc01c6a0 con 0x7f460c101fc0 2024-03-21T12:51:30.603 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.602+0000 7f46097fa700 1 -- 172.21.15.155:0/215606841 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(59..59 src has 1..59) v4 ==== 4081+0+0 (unknown 972769531 0 0) 0x7f45fc088fe0 con 0x7f460c101fc0 2024-03-21T12:51:30.606 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.606+0000 7f46097fa700 1 -- 172.21.15.155:0/215606841 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f45fc056e90 con 0x7f460c101fc0 2024-03-21T12:51:30.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.811+0000 7f4611f4e700 1 -- 172.21.15.155:0/215606841 --> v1:172.21.15.43:6800/4081968544 -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi043=foo.a", "target": ["mon-mgr", ""]}) v1 -- 0x7f45ec000bc0 con 0x7f45f806f090 2024-03-21T12:51:30.818 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.818+0000 7f46097fa700 1 -- 172.21.15.155:0/215606841 <== mgr.14152 v1:172.21.15.43:6800/4081968544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (unknown 0 0 1123153589) 0x7f45ec000bc0 con 0x7f45f806f090 2024-03-21T12:51:30.818 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled rgw.foo.a update... 2024-03-21T12:51:30.820 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.820+0000 7f4611f4e700 1 -- 172.21.15.155:0/215606841 >> v1:172.21.15.43:6800/4081968544 conn(0x7f45f806f090 legacy=0x7f45f8071550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:30.820 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.820+0000 7f4611f4e700 1 -- 172.21.15.155:0/215606841 >> v1:172.21.15.155:6789/0 conn(0x7f460c101fc0 legacy=0x7f460c197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:30.820 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.820+0000 7f4611f4e700 1 -- 172.21.15.155:0/215606841 shutdown_connections 2024-03-21T12:51:30.820 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.820+0000 7f4611f4e700 1 -- 172.21.15.155:0/215606841 >> 172.21.15.155:0/215606841 conn(0x7f460c0fd560 msgr2=0x7f460c1096e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:51:30.821 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.821+0000 7f4611f4e700 1 -- 172.21.15.155:0/215606841 shutdown_connections 2024-03-21T12:51:30.821 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:30.821+0000 7f4611f4e700 1 -- 172.21.15.155:0/215606841 wait complete. 2024-03-21T12:51:31.489 DEBUG:teuthology.orchestra.run.smithi043:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@rgw.foo.a.service 2024-03-21T12:51:31.492 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi155 2024-03-21T12:51:31.492 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph osd pool create datapool 3 3 replicated 2024-03-21T12:51:31.728 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T12:51:32.074 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:31 smithi043 ceph-mon[30062]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-21T12:51:32.074 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:31 smithi043 ceph-mon[30062]: from='client.24349 v1:172.21.15.155:0/215606841' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi043=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:51:32.074 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:31 smithi043 ceph-mon[30062]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-03-21T12:51:32.074 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:31 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:32.074 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:31 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:51:32.074 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:31 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:32.074 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:31 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:51:32.074 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:31 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:32.074 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:31 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-21T12:51:32.075 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:31 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-21T12:51:32.075 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:31 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:32.075 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:31 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:32.075 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:31 smithi043 ceph-mon[41699]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-21T12:51:32.075 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:31 smithi043 ceph-mon[41699]: from='client.24349 v1:172.21.15.155:0/215606841' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi043=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:51:32.075 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:31 smithi043 ceph-mon[41699]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-03-21T12:51:32.075 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:31 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:32.075 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:31 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:51:32.075 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:31 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:32.075 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:31 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:51:32.075 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:31 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:32.075 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:31 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-21T12:51:32.076 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:31 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-21T12:51:32.076 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:31 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:32.076 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:31 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:32.097 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:31 smithi155 ceph-mon[33846]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-03-21T12:51:32.097 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:31 smithi155 ceph-mon[33846]: from='client.24349 v1:172.21.15.155:0/215606841' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi043=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:51:32.097 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:31 smithi155 ceph-mon[33846]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-03-21T12:51:32.097 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:31 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:32.097 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:31 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:51:32.098 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:31 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:32.098 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:31 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:51:32.098 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:31 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:32.098 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:31 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-21T12:51:32.098 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:31 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-21T12:51:32.098 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:31 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:32.098 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:31 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:33.134 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:32 smithi155 ceph-mon[33846]: Deploying daemon rgw.foo.a on smithi043 2024-03-21T12:51:33.197 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:32 smithi043 ceph-mon[30062]: Deploying daemon rgw.foo.a on smithi043 2024-03-21T12:51:33.197 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:32 smithi043 ceph-mon[41699]: Deploying daemon rgw.foo.a on smithi043 2024-03-21T12:51:33.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.537+0000 7febbdbe6700 1 -- 172.21.15.155:0/1658037335 >> v1:172.21.15.155:6789/0 conn(0x7febb8106490 legacy=0x7febb8108880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:33.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.537+0000 7febbdbe6700 1 -- 172.21.15.155:0/1658037335 shutdown_connections 2024-03-21T12:51:33.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.537+0000 7febbdbe6700 1 -- 172.21.15.155:0/1658037335 >> 172.21.15.155:0/1658037335 conn(0x7febb80fbc00 msgr2=0x7febb80fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:51:33.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.538+0000 7febbdbe6700 1 -- 172.21.15.155:0/1658037335 shutdown_connections 2024-03-21T12:51:33.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.538+0000 7febbdbe6700 1 -- 172.21.15.155:0/1658037335 wait complete. 2024-03-21T12:51:33.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.538+0000 7febbdbe6700 1 Processor -- start 2024-03-21T12:51:33.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.538+0000 7febbdbe6700 1 -- start start 2024-03-21T12:51:33.540 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.538+0000 7febbdbe6700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7febb8197cc0 con 0x7febb8106490 2024-03-21T12:51:33.541 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.538+0000 7febbdbe6700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7febb81a34b0 con 0x7febb8100280 2024-03-21T12:51:33.541 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.538+0000 7febbdbe6700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7febb81a4690 con 0x7febb810a0f0 2024-03-21T12:51:33.541 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.538+0000 7febb77fe700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7febb8100280 0x7febb8197170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:35176/0 (socket says 172.21.15.155:35176) 2024-03-21T12:51:33.541 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.539+0000 7febb77fe700 1 -- 172.21.15.155:0/981943946 learned_addr learned my addr 172.21.15.155:0/981943946 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-03-21T12:51:33.541 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.539+0000 7febb4ff9700 1 -- 172.21.15.155:0/981943946 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2417449576 0 0) 0x7febb81a34b0 con 0x7febb8100280 2024-03-21T12:51:33.541 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.539+0000 7febb4ff9700 1 -- 172.21.15.155:0/981943946 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feba00035b0 con 0x7febb8100280 2024-03-21T12:51:33.541 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.539+0000 7febb4ff9700 1 -- 172.21.15.155:0/981943946 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663258535 0 0) 0x7febb8197cc0 con 0x7febb8106490 2024-03-21T12:51:33.541 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.539+0000 7febb4ff9700 1 -- 172.21.15.155:0/981943946 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febb81a34b0 con 0x7febb8106490 2024-03-21T12:51:33.541 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.539+0000 7febb4ff9700 1 -- 172.21.15.155:0/981943946 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1125672881 0 0) 0x7febb81a4690 con 0x7febb810a0f0 2024-03-21T12:51:33.541 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.539+0000 7febb4ff9700 1 -- 172.21.15.155:0/981943946 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febb8197cc0 con 0x7febb810a0f0 2024-03-21T12:51:33.541 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.539+0000 7febb4ff9700 1 -- 172.21.15.155:0/981943946 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2616324087 0 0) 0x7feba00035b0 con 0x7febb8100280 2024-03-21T12:51:33.541 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.539+0000 7febb4ff9700 1 -- 172.21.15.155:0/981943946 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7febb81a4690 con 0x7febb8100280 2024-03-21T12:51:33.542 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.539+0000 7febb4ff9700 1 -- 172.21.15.155:0/981943946 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7feba8004060 con 0x7febb8100280 2024-03-21T12:51:33.542 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.539+0000 7febb4ff9700 1 -- 172.21.15.155:0/981943946 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1442425176 0 0) 0x7febb81a34b0 con 0x7febb8106490 2024-03-21T12:51:33.542 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.539+0000 7febb4ff9700 1 -- 172.21.15.155:0/981943946 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feba00035b0 con 0x7febb8106490 2024-03-21T12:51:33.542 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.539+0000 7febb4ff9700 1 -- 172.21.15.155:0/981943946 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7febac004740 con 0x7febb8106490 2024-03-21T12:51:33.542 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.539+0000 7febb4ff9700 1 -- 172.21.15.155:0/981943946 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2224372289 0 0) 0x7febb8197cc0 con 0x7febb810a0f0 2024-03-21T12:51:33.542 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.539+0000 7febb4ff9700 1 -- 172.21.15.155:0/981943946 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7febb81a34b0 con 0x7febb810a0f0 2024-03-21T12:51:33.542 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.539+0000 7febb4ff9700 1 -- 172.21.15.155:0/981943946 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3972183638 0 0) 0x7febb81a4690 con 0x7febb8100280 2024-03-21T12:51:33.542 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.539+0000 7febb4ff9700 1 -- 172.21.15.155:0/981943946 >> v1:172.21.15.43:6790/0 conn(0x7febb810a0f0 legacy=0x7febb81a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:33.542 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.539+0000 7febb4ff9700 1 -- 172.21.15.155:0/981943946 >> v1:172.21.15.43:6789/0 conn(0x7febb8106490 legacy=0x7febb819d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:33.542 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.539+0000 7febb4ff9700 1 -- 172.21.15.155:0/981943946 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb81a58b0 con 0x7febb8100280 2024-03-21T12:51:33.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.539+0000 7febbdbe6700 1 -- 172.21.15.155:0/981943946 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb81a2500 con 0x7febb8100280 2024-03-21T12:51:33.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.539+0000 7febbdbe6700 1 -- 172.21.15.155:0/981943946 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7febb81a29c0 con 0x7febb8100280 2024-03-21T12:51:33.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.539+0000 7febb4ff9700 1 -- 172.21.15.155:0/981943946 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7feba8001360 con 0x7febb8100280 2024-03-21T12:51:33.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.539+0000 7febb4ff9700 1 -- 172.21.15.155:0/981943946 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7feba8005c40 con 0x7febb8100280 2024-03-21T12:51:33.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.540+0000 7febbdbe6700 1 -- 172.21.15.155:0/981943946 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba4005120 con 0x7febb8100280 2024-03-21T12:51:33.543 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.541+0000 7febb4ff9700 1 -- 172.21.15.155:0/981943946 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 15) v1 ==== 92095+0+0 (unknown 2546926431 0 0) 0x7feba8000ed0 con 0x7febb8100280 2024-03-21T12:51:33.544 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.541+0000 7febb4ff9700 1 -- 172.21.15.155:0/981943946 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(59..59 src has 1..59) v4 ==== 4081+0+0 (unknown 972769531 0 0) 0x7feba8088dd0 con 0x7febb8100280 2024-03-21T12:51:33.545 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.546+0000 7febb4ff9700 1 -- 172.21.15.155:0/981943946 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7feba8005c20 con 0x7febb8100280 2024-03-21T12:51:33.742 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:33.743+0000 7febbdbe6700 1 -- 172.21.15.155:0/981943946 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) v1 -- 0x7feba4005410 con 0x7febb8100280 2024-03-21T12:51:33.892 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Mar 21 12:51:33 smithi043 systemd[1]: Starting Ceph rgw.foo.a for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T12:51:34.052 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:34.052+0000 7febb4ff9700 1 -- 172.21.15.155:0/981943946 <== mon.1 v1:172.21.15.155:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'datapool' created v60) v1 ==== 160+0+0 (unknown 701681216 0 0) 0x7feba8056c80 con 0x7febb8100280 2024-03-21T12:51:34.054 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:34.054+0000 7febbdbe6700 1 -- 172.21.15.155:0/981943946 >> v1:172.21.15.43:6800/4081968544 conn(0x7feba006f040 legacy=0x7feba0071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:34.054 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:34.055+0000 7febbdbe6700 1 -- 172.21.15.155:0/981943946 >> v1:172.21.15.155:6789/0 conn(0x7febb8100280 legacy=0x7febb8197170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:34.055 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:34.055+0000 7febbdbe6700 1 -- 172.21.15.155:0/981943946 shutdown_connections 2024-03-21T12:51:34.055 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:34.055+0000 7febbdbe6700 1 -- 172.21.15.155:0/981943946 >> 172.21.15.155:0/981943946 conn(0x7febb80fbc00 msgr2=0x7febb80fbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:51:34.055 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:34.055+0000 7febbdbe6700 1 -- 172.21.15.155:0/981943946 shutdown_connections 2024-03-21T12:51:34.055 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:34.055+0000 7febbdbe6700 1 -- 172.21.15.155:0/981943946 wait complete. 2024-03-21T12:51:34.055 INFO:teuthology.orchestra.run.smithi155.stderr:pool 'datapool' created 2024-03-21T12:51:34.169 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:33 smithi043 ceph-mon[30062]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-21T12:51:34.169 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:33 smithi043 ceph-mon[41699]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-21T12:51:34.170 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Mar 21 12:51:33 smithi043 podman[69832]: 2024-03-21 12:51:33.796094911 +0000 UTC m=+0.018819697 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93 2024-03-21T12:51:34.170 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Mar 21 12:51:33 smithi043 podman[69832]: 2024-03-21 12:51:33.916584172 +0000 UTC m=+0.139308935 container create 04f8420e74d02a4ad112aaf4ffe81e89f06b468007a81263a56603e642a04886 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-rgw-foo-a, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T12:51:34.170 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Mar 21 12:51:34 smithi043 podman[69832]: 2024-03-21 12:51:34.169545629 +0000 UTC m=+0.392270387 container init 04f8420e74d02a4ad112aaf4ffe81e89f06b468007a81263a56603e642a04886 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-rgw-foo-a, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T12:51:34.357 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:34 smithi155 ceph-mon[33846]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-03-21T12:51:34.483 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Mar 21 12:51:34 smithi043 podman[69832]: 2024-03-21 12:51:34.174789692 +0000 UTC m=+0.397514452 container start 04f8420e74d02a4ad112aaf4ffe81e89f06b468007a81263a56603e642a04886 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-rgw-foo-a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, ceph=True) 2024-03-21T12:51:34.483 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Mar 21 12:51:34 smithi043 bash[69832]: 04f8420e74d02a4ad112aaf4ffe81e89f06b468007a81263a56603e642a04886 2024-03-21T12:51:34.484 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Mar 21 12:51:34 smithi043 systemd[1]: Started Ceph rgw.foo.a for c06c8d1a-e780-11ee-95cd-87774f69a715. 2024-03-21T12:51:34.726 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- rbd pool init datapool 2024-03-21T12:51:34.971 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T12:51:35.264 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:35 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.155:0/981943946' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-21T12:51:35.264 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:35 smithi155 ceph-mon[33846]: from='client.24355 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-21T12:51:35.264 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:35 smithi155 ceph-mon[33846]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T12:51:35.264 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:35 smithi155 ceph-mon[33846]: from='client.24355 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-03-21T12:51:35.264 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:35 smithi155 ceph-mon[33846]: osdmap e60: 8 total, 8 up, 8 in 2024-03-21T12:51:35.264 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:35 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:35.264 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:35 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:35.264 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:35 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:35.264 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:35 smithi155 ceph-mon[33846]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-03-21T12:51:35.265 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:35 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:35.265 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:35 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:35.265 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:35 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:51:35.400 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:35 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.155:0/981943946' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-21T12:51:35.400 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:35 smithi043 ceph-mon[41699]: from='client.24355 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-21T12:51:35.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:35 smithi043 ceph-mon[41699]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T12:51:35.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:35 smithi043 ceph-mon[41699]: from='client.24355 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-03-21T12:51:35.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:35 smithi043 ceph-mon[41699]: osdmap e60: 8 total, 8 up, 8 in 2024-03-21T12:51:35.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:35 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:35.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:35 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:35.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:35 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:35.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:35 smithi043 ceph-mon[41699]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-03-21T12:51:35.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:35 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:35.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:35 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:35.401 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:35 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:51:35.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:35 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.155:0/981943946' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-21T12:51:35.401 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:35 smithi043 ceph-mon[30062]: from='client.24355 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-21T12:51:35.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:35 smithi043 ceph-mon[30062]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-21T12:51:35.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:35 smithi043 ceph-mon[30062]: from='client.24355 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-03-21T12:51:35.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:35 smithi043 ceph-mon[30062]: osdmap e60: 8 total, 8 up, 8 in 2024-03-21T12:51:35.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:35 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:35.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:35 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:35.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:35 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:35.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:35 smithi043 ceph-mon[30062]: Saving service rgw.foo.a spec with placement smithi043=foo.a;count:1 2024-03-21T12:51:35.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:35 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:35.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:35 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:35.402 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:35 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:51:36.155 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:36 smithi155 ceph-mon[33846]: osdmap e61: 8 total, 8 up, 8 in 2024-03-21T12:51:36.155 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2049702790' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-21T12:51:36.155 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:36 smithi155 ceph-mon[33846]: from='client.34358 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-21T12:51:36.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:36 smithi043 ceph-mon[30062]: osdmap e61: 8 total, 8 up, 8 in 2024-03-21T12:51:36.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2049702790' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-21T12:51:36.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:36 smithi043 ceph-mon[30062]: from='client.34358 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-21T12:51:36.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:36 smithi043 ceph-mon[41699]: osdmap e61: 8 total, 8 up, 8 in 2024-03-21T12:51:36.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2049702790' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-21T12:51:36.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:36 smithi043 ceph-mon[41699]: from='client.34358 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-21T12:51:37.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:37 smithi043 ceph-mon[30062]: pgmap v193: 36 pgs: 6 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:51:37.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:37 smithi043 ceph-mon[30062]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:51:37.386 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:37 smithi043 ceph-mon[30062]: from='client.34358 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-03-21T12:51:37.386 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:37 smithi043 ceph-mon[30062]: osdmap e62: 8 total, 8 up, 8 in 2024-03-21T12:51:37.386 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:37 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:37.386 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:37 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:37.386 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:37 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:37.386 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:37 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:51:37.386 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:37 smithi043 ceph-mon[30062]: Checking dashboard <-> RGW credentials 2024-03-21T12:51:37.386 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:37 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.155:0/1654316091' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-21T12:51:37.386 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:37 smithi043 ceph-mon[30062]: from='client.24406 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-21T12:51:37.386 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:37 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:37.386 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:37 smithi043 ceph-mon[41699]: pgmap v193: 36 pgs: 6 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:51:37.386 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:37 smithi043 ceph-mon[41699]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:51:37.387 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:37 smithi043 ceph-mon[41699]: from='client.34358 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-03-21T12:51:37.387 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:37 smithi043 ceph-mon[41699]: osdmap e62: 8 total, 8 up, 8 in 2024-03-21T12:51:37.387 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:37.387 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:37.387 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:37.387 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:51:37.387 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:37 smithi043 ceph-mon[41699]: Checking dashboard <-> RGW credentials 2024-03-21T12:51:37.387 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:37 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.155:0/1654316091' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-21T12:51:37.387 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:37 smithi043 ceph-mon[41699]: from='client.24406 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-21T12:51:37.387 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:37 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:37.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:37 smithi155 ceph-mon[33846]: pgmap v193: 36 pgs: 6 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:51:37.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:37 smithi155 ceph-mon[33846]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:51:37.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:37 smithi155 ceph-mon[33846]: from='client.34358 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-03-21T12:51:37.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:37 smithi155 ceph-mon[33846]: osdmap e62: 8 total, 8 up, 8 in 2024-03-21T12:51:37.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:37 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:37.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:37 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:37.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:37 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:37.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:37 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:51:37.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:37 smithi155 ceph-mon[33846]: Checking dashboard <-> RGW credentials 2024-03-21T12:51:37.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:37 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.155:0/1654316091' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-21T12:51:37.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:37 smithi155 ceph-mon[33846]: from='client.24406 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-21T12:51:37.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:37 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:38.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:38 smithi043 ceph-mon[30062]: from='client.24406 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-03-21T12:51:38.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:38 smithi043 ceph-mon[30062]: osdmap e63: 8 total, 8 up, 8 in 2024-03-21T12:51:38.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:38 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1110778609' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-21T12:51:38.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:38 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1999109308' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-21T12:51:38.396 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:38 smithi043 ceph-mon[30062]: from='client.34397 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-21T12:51:38.396 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:38 smithi043 ceph-mon[41699]: from='client.24406 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-03-21T12:51:38.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:38 smithi043 ceph-mon[41699]: osdmap e63: 8 total, 8 up, 8 in 2024-03-21T12:51:38.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:38 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1110778609' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-21T12:51:38.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:38 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1999109308' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-21T12:51:38.397 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:38 smithi043 ceph-mon[41699]: from='client.34397 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-21T12:51:38.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:38 smithi155 ceph-mon[33846]: from='client.24406 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-03-21T12:51:38.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:38 smithi155 ceph-mon[33846]: osdmap e63: 8 total, 8 up, 8 in 2024-03-21T12:51:38.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:38 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1110778609' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-21T12:51:38.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:38 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1999109308' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-21T12:51:38.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:38 smithi155 ceph-mon[33846]: from='client.34397 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-21T12:51:39.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:39 smithi155 ceph-mon[33846]: pgmap v196: 68 pgs: 16 active+clean, 15 creating+peering, 37 unknown; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 511 B/s wr, 0 op/s 2024-03-21T12:51:39.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:39 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1110778609' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-03-21T12:51:39.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:39 smithi155 ceph-mon[33846]: from='client.34397 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-03-21T12:51:39.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:39 smithi155 ceph-mon[33846]: osdmap e64: 8 total, 8 up, 8 in 2024-03-21T12:51:39.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:39 smithi043 ceph-mon[30062]: pgmap v196: 68 pgs: 16 active+clean, 15 creating+peering, 37 unknown; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 511 B/s wr, 0 op/s 2024-03-21T12:51:39.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:39 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1110778609' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-03-21T12:51:39.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:39 smithi043 ceph-mon[30062]: from='client.34397 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-03-21T12:51:39.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:39 smithi043 ceph-mon[30062]: osdmap e64: 8 total, 8 up, 8 in 2024-03-21T12:51:39.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:39 smithi043 ceph-mon[41699]: pgmap v196: 68 pgs: 16 active+clean, 15 creating+peering, 37 unknown; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 511 B/s wr, 0 op/s 2024-03-21T12:51:39.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:39 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1110778609' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-03-21T12:51:39.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:39 smithi043 ceph-mon[41699]: from='client.34397 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-03-21T12:51:39.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:39 smithi043 ceph-mon[41699]: osdmap e64: 8 total, 8 up, 8 in 2024-03-21T12:51:39.839 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.155 --placement '1;smithi155=iscsi.a' 2024-03-21T12:51:40.097 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T12:51:40.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:40 smithi155 ceph-mon[33846]: osdmap e65: 8 total, 8 up, 8 in 2024-03-21T12:51:40.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:40 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1999109308' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-21T12:51:40.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:40 smithi155 ceph-mon[33846]: from='client.34397 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-21T12:51:40.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:40 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1110778609' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-21T12:51:40.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:40 smithi155 ceph-mon[33846]: from='client.34397 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-03-21T12:51:40.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:40 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1110778609' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-03-21T12:51:40.374 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:40 smithi155 ceph-mon[33846]: osdmap e66: 8 total, 8 up, 8 in 2024-03-21T12:51:40.483 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:40 smithi043 ceph-mon[41699]: osdmap e65: 8 total, 8 up, 8 in 2024-03-21T12:51:40.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:40 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1999109308' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-21T12:51:40.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:40 smithi043 ceph-mon[41699]: from='client.34397 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-21T12:51:40.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:40 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1110778609' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-21T12:51:40.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:40 smithi043 ceph-mon[41699]: from='client.34397 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-03-21T12:51:40.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:40 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1110778609' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-03-21T12:51:40.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:40 smithi043 ceph-mon[41699]: osdmap e66: 8 total, 8 up, 8 in 2024-03-21T12:51:40.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:40 smithi043 ceph-mon[30062]: osdmap e65: 8 total, 8 up, 8 in 2024-03-21T12:51:40.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:40 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1999109308' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-21T12:51:40.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:40 smithi043 ceph-mon[30062]: from='client.34397 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-21T12:51:40.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:40 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1110778609' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-21T12:51:40.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:40 smithi043 ceph-mon[30062]: from='client.34397 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-03-21T12:51:40.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:40 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1110778609' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-03-21T12:51:40.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:40 smithi043 ceph-mon[30062]: osdmap e66: 8 total, 8 up, 8 in 2024-03-21T12:51:41.301 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:41 smithi155 ceph-mon[33846]: pgmap v199: 100 pgs: 37 active+clean, 24 creating+peering, 39 unknown; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 1023 B/s wr, 1 op/s 2024-03-21T12:51:41.301 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:41 smithi155 ceph-mon[33846]: osdmap e67: 8 total, 8 up, 8 in 2024-03-21T12:51:41.301 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1110778609' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-21T12:51:41.301 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1999109308' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-21T12:51:41.302 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:41 smithi155 ceph-mon[33846]: from='client.34397 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-21T12:51:41.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:41 smithi043 ceph-mon[30062]: pgmap v199: 100 pgs: 37 active+clean, 24 creating+peering, 39 unknown; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 1023 B/s wr, 1 op/s 2024-03-21T12:51:41.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:41 smithi043 ceph-mon[30062]: osdmap e67: 8 total, 8 up, 8 in 2024-03-21T12:51:41.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1110778609' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-21T12:51:41.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1999109308' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-21T12:51:41.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:41 smithi043 ceph-mon[30062]: from='client.34397 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-21T12:51:41.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:41 smithi043 ceph-mon[41699]: pgmap v199: 100 pgs: 37 active+clean, 24 creating+peering, 39 unknown; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 1023 B/s wr, 1 op/s 2024-03-21T12:51:41.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:41 smithi043 ceph-mon[41699]: osdmap e67: 8 total, 8 up, 8 in 2024-03-21T12:51:41.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1110778609' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-21T12:51:41.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1999109308' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-21T12:51:41.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:41 smithi043 ceph-mon[41699]: from='client.34397 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-21T12:51:41.798 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.798+0000 7f593afe9700 1 -- 172.21.15.155:0/3765992267 >> v1:172.21.15.155:6789/0 conn(0x7f593410a0b0 legacy=0x7f593410c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:41.798 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.798+0000 7f593afe9700 1 -- 172.21.15.155:0/3765992267 shutdown_connections 2024-03-21T12:51:41.799 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.798+0000 7f593afe9700 1 -- 172.21.15.155:0/3765992267 >> 172.21.15.155:0/3765992267 conn(0x7f59340fc450 msgr2=0x7f59340fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:51:41.799 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.798+0000 7f593afe9700 1 -- 172.21.15.155:0/3765992267 shutdown_connections 2024-03-21T12:51:41.799 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.798+0000 7f593afe9700 1 -- 172.21.15.155:0/3765992267 wait complete. 2024-03-21T12:51:41.799 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.798+0000 7f593afe9700 1 Processor -- start 2024-03-21T12:51:41.799 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.798+0000 7f593afe9700 1 -- start start 2024-03-21T12:51:41.799 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.799+0000 7f593afe9700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f59341980f0 con 0x7f5934106510 2024-03-21T12:51:41.799 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.799+0000 7f593afe9700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f59341a38e0 con 0x7f593410a0b0 2024-03-21T12:51:41.799 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.799+0000 7f593afe9700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f59341a4ac0 con 0x7f5934102a00 2024-03-21T12:51:41.799 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.799+0000 7f5939586700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f593410a0b0 0x7f59341a11b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:53996/0 (socket says 172.21.15.155:53996) 2024-03-21T12:51:41.799 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.799+0000 7f5939586700 1 -- 172.21.15.155:0/651121706 learned_addr learned my addr 172.21.15.155:0/651121706 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-03-21T12:51:41.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.799+0000 7f5931ffb700 1 -- 172.21.15.155:0/651121706 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3879147039 0 0) 0x7f59341a38e0 con 0x7f593410a0b0 2024-03-21T12:51:41.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.799+0000 7f5931ffb700 1 -- 172.21.15.155:0/651121706 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f591c0035b0 con 0x7f593410a0b0 2024-03-21T12:51:41.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.799+0000 7f5931ffb700 1 -- 172.21.15.155:0/651121706 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 145441357 0 0) 0x7f59341a4ac0 con 0x7f5934102a00 2024-03-21T12:51:41.801 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.799+0000 7f5931ffb700 1 -- 172.21.15.155:0/651121706 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59341a38e0 con 0x7f5934102a00 2024-03-21T12:51:41.802 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.799+0000 7f5931ffb700 1 -- 172.21.15.155:0/651121706 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4167206173 0 0) 0x7f59341980f0 con 0x7f5934106510 2024-03-21T12:51:41.802 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.799+0000 7f5931ffb700 1 -- 172.21.15.155:0/651121706 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59341a4ac0 con 0x7f5934106510 2024-03-21T12:51:41.802 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.799+0000 7f5931ffb700 1 -- 172.21.15.155:0/651121706 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3144309273 0 0) 0x7f591c0035b0 con 0x7f593410a0b0 2024-03-21T12:51:41.802 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.799+0000 7f5931ffb700 1 -- 172.21.15.155:0/651121706 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f59341980f0 con 0x7f593410a0b0 2024-03-21T12:51:41.802 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.799+0000 7f5931ffb700 1 -- 172.21.15.155:0/651121706 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f5924001430 con 0x7f593410a0b0 2024-03-21T12:51:41.802 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.799+0000 7f5931ffb700 1 -- 172.21.15.155:0/651121706 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1878156072 0 0) 0x7f59341a38e0 con 0x7f5934102a00 2024-03-21T12:51:41.802 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.799+0000 7f5931ffb700 1 -- 172.21.15.155:0/651121706 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f591c0035b0 con 0x7f5934102a00 2024-03-21T12:51:41.802 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.799+0000 7f5931ffb700 1 -- 172.21.15.155:0/651121706 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f592c001b60 con 0x7f5934102a00 2024-03-21T12:51:41.802 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.799+0000 7f5931ffb700 1 -- 172.21.15.155:0/651121706 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 197750311 0 0) 0x7f59341a4ac0 con 0x7f5934106510 2024-03-21T12:51:41.802 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.799+0000 7f5931ffb700 1 -- 172.21.15.155:0/651121706 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f59341a38e0 con 0x7f5934106510 2024-03-21T12:51:41.802 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.799+0000 7f5931ffb700 1 -- 172.21.15.155:0/651121706 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4283982815 0 0) 0x7f59341980f0 con 0x7f593410a0b0 2024-03-21T12:51:41.802 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.799+0000 7f5931ffb700 1 -- 172.21.15.155:0/651121706 >> v1:172.21.15.43:6790/0 conn(0x7f5934102a00 legacy=0x7f59341975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:41.802 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.799+0000 7f5931ffb700 1 -- 172.21.15.155:0/651121706 >> v1:172.21.15.43:6789/0 conn(0x7f5934106510 legacy=0x7f593419da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:41.802 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.799+0000 7f5931ffb700 1 -- 172.21.15.155:0/651121706 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59341a5ce0 con 0x7f593410a0b0 2024-03-21T12:51:41.803 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.800+0000 7f593afe9700 1 -- 172.21.15.155:0/651121706 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59341a2930 con 0x7f593410a0b0 2024-03-21T12:51:41.803 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.800+0000 7f5931ffb700 1 -- 172.21.15.155:0/651121706 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f59240016e0 con 0x7f593410a0b0 2024-03-21T12:51:41.803 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.800+0000 7f5931ffb700 1 -- 172.21.15.155:0/651121706 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f59240050f0 con 0x7f593410a0b0 2024-03-21T12:51:41.803 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.800+0000 7f593afe9700 1 -- 172.21.15.155:0/651121706 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f59341a2e70 con 0x7f593410a0b0 2024-03-21T12:51:41.803 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.801+0000 7f593afe9700 1 -- 172.21.15.155:0/651121706 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5934068a80 con 0x7f593410a0b0 2024-03-21T12:51:41.803 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.801+0000 7f5931ffb700 1 -- 172.21.15.155:0/651121706 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 15) v1 ==== 92095+0+0 (unknown 2546926431 0 0) 0x7f5924003ab0 con 0x7f593410a0b0 2024-03-21T12:51:41.803 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.802+0000 7f5931ffb700 1 -- 172.21.15.155:0/651121706 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(67..67 src has 1..67) v4 ==== 5915+0+0 (unknown 4253809863 0 0) 0x7f5924089850 con 0x7f593410a0b0 2024-03-21T12:51:41.806 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:41.806+0000 7f5931ffb700 1 -- 172.21.15.155:0/651121706 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f5924056fe0 con 0x7f593410a0b0 2024-03-21T12:51:42.017 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:42.017+0000 7f593afe9700 1 -- 172.21.15.155:0/651121706 --> v1:172.21.15.43:6800/4081968544 -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.155", "placement": "1;smithi155=iscsi.a", "target": ["mon-mgr", ""]}) v1 -- 0x7f59341a3200 con 0x7f591c06f060 2024-03-21T12:51:42.025 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:42.025+0000 7f5931ffb700 1 -- 172.21.15.155:0/651121706 <== mgr.14152 v1:172.21.15.43:6800/4081968544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+35 (unknown 0 0 803663096) 0x7f59341a3200 con 0x7f591c06f060 2024-03-21T12:51:42.025 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled iscsi.datapool update... 2024-03-21T12:51:42.027 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:42.027+0000 7f593afe9700 1 -- 172.21.15.155:0/651121706 >> v1:172.21.15.43:6800/4081968544 conn(0x7f591c06f060 legacy=0x7f591c071520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:42.027 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:42.027+0000 7f593afe9700 1 -- 172.21.15.155:0/651121706 >> v1:172.21.15.155:6789/0 conn(0x7f593410a0b0 legacy=0x7f59341a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:42.027 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:42.027+0000 7f593afe9700 1 -- 172.21.15.155:0/651121706 shutdown_connections 2024-03-21T12:51:42.028 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:42.027+0000 7f593afe9700 1 -- 172.21.15.155:0/651121706 >> 172.21.15.155:0/651121706 conn(0x7f59340fc450 msgr2=0x7f593410c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:51:42.028 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:42.028+0000 7f593afe9700 1 -- 172.21.15.155:0/651121706 shutdown_connections 2024-03-21T12:51:42.028 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:42.028+0000 7f593afe9700 1 -- 172.21.15.155:0/651121706 wait complete. 2024-03-21T12:51:42.349 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:42 smithi155 ceph-mon[33846]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:51:42.349 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:42 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:42.349 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1110778609' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-21T12:51:42.350 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:42 smithi155 ceph-mon[33846]: from='client.34397 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-21T12:51:42.350 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:42 smithi155 ceph-mon[33846]: osdmap e68: 8 total, 8 up, 8 in 2024-03-21T12:51:42.350 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1110778609' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-21T12:51:42.350 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1999109308' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-21T12:51:42.350 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:42 smithi155 ceph-mon[33846]: from='client.34397 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-21T12:51:42.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:42 smithi043 ceph-mon[30062]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:51:42.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:42 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:42.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1110778609' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-21T12:51:42.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:42 smithi043 ceph-mon[30062]: from='client.34397 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-21T12:51:42.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:42 smithi043 ceph-mon[30062]: osdmap e68: 8 total, 8 up, 8 in 2024-03-21T12:51:42.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1110778609' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-21T12:51:42.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1999109308' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-21T12:51:42.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:42 smithi043 ceph-mon[30062]: from='client.34397 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-21T12:51:42.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:42 smithi043 ceph-mon[41699]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:51:42.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:42 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:42.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1110778609' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-21T12:51:42.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:42 smithi043 ceph-mon[41699]: from='client.34397 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-21T12:51:42.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:42 smithi043 ceph-mon[41699]: osdmap e68: 8 total, 8 up, 8 in 2024-03-21T12:51:42.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1110778609' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-21T12:51:42.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1999109308' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-21T12:51:42.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:42 smithi043 ceph-mon[41699]: from='client.34397 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-21T12:51:42.659 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-03-21T12:51:42.659 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-21T12:51:42.659 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-03-21T12:51:42.695 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-21T12:51:42.695 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-03-21T12:51:42.724 DEBUG:teuthology.orchestra.run.smithi155:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@iscsi.iscsi.a.service 2024-03-21T12:51:42.767 INFO:tasks.cephadm:Adding prometheus.a on smithi155 2024-03-21T12:51:42.767 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph orch apply prometheus '1;smithi155=a' 2024-03-21T12:51:43.042 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T12:51:43.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:43 smithi155 ceph-mon[33846]: pgmap v202: 132 pgs: 72 active+clean, 35 creating+peering, 25 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-21T12:51:43.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:43 smithi155 ceph-mon[33846]: from='client.24412 v1:172.21.15.155:0/651121706' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.155", "placement": "1;smithi155=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:51:43.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:43 smithi155 ceph-mon[33846]: Saving service iscsi.datapool spec with placement smithi155=iscsi.a;count:1 2024-03-21T12:51:43.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1110778609' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-21T12:51:43.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:43 smithi155 ceph-mon[33846]: from='client.34397 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-21T12:51:43.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:43 smithi155 ceph-mon[33846]: osdmap e69: 8 total, 8 up, 8 in 2024-03-21T12:51:43.483 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:43 smithi043 ceph-mon[41699]: pgmap v202: 132 pgs: 72 active+clean, 35 creating+peering, 25 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-21T12:51:43.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:43 smithi043 ceph-mon[41699]: from='client.24412 v1:172.21.15.155:0/651121706' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.155", "placement": "1;smithi155=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:51:43.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:43 smithi043 ceph-mon[41699]: Saving service iscsi.datapool spec with placement smithi155=iscsi.a;count:1 2024-03-21T12:51:43.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1110778609' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-21T12:51:43.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:43 smithi043 ceph-mon[41699]: from='client.34397 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-21T12:51:43.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:43 smithi043 ceph-mon[41699]: osdmap e69: 8 total, 8 up, 8 in 2024-03-21T12:51:43.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:43 smithi043 ceph-mon[30062]: pgmap v202: 132 pgs: 72 active+clean, 35 creating+peering, 25 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-21T12:51:43.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:43 smithi043 ceph-mon[30062]: from='client.24412 v1:172.21.15.155:0/651121706' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.155", "placement": "1;smithi155=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:51:43.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:43 smithi043 ceph-mon[30062]: Saving service iscsi.datapool spec with placement smithi155=iscsi.a;count:1 2024-03-21T12:51:43.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1110778609' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-21T12:51:43.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:43 smithi043 ceph-mon[30062]: from='client.34397 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-21T12:51:43.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:43 smithi043 ceph-mon[30062]: osdmap e69: 8 total, 8 up, 8 in 2024-03-21T12:51:44.600 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:44 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:44.600 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:44 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:44.600 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:44 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:44.600 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:44 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-21T12:51:44.600 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:44 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-03-21T12:51:44.600 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:44 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:44.600 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:44 smithi155 ceph-mon[33846]: Deploying daemon iscsi.iscsi.a on smithi155 2024-03-21T12:51:44.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:44 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:44.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:44 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:44.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:44 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:44.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:44 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-21T12:51:44.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:44 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-03-21T12:51:44.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:44 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:44.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:44 smithi043 ceph-mon[30062]: Deploying daemon iscsi.iscsi.a on smithi155 2024-03-21T12:51:44.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:44 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:44.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:44 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:44.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:44 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:44.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:44 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-21T12:51:44.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:44 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-03-21T12:51:44.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:44 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:44.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:44 smithi043 ceph-mon[41699]: Deploying daemon iscsi.iscsi.a on smithi155 2024-03-21T12:51:45.513 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.509+0000 7fa99eb58700 1 -- 172.21.15.155:0/598477172 >> v1:172.21.15.155:6789/0 conn(0x7fa998101fa0 legacy=0x7fa9981023c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:45.513 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.509+0000 7fa99eb58700 1 -- 172.21.15.155:0/598477172 shutdown_connections 2024-03-21T12:51:45.514 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.509+0000 7fa99eb58700 1 -- 172.21.15.155:0/598477172 >> 172.21.15.155:0/598477172 conn(0x7fa9980fd540 msgr2=0x7fa9980ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:51:45.514 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.509+0000 7fa99eb58700 1 -- 172.21.15.155:0/598477172 shutdown_connections 2024-03-21T12:51:45.514 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.509+0000 7fa99eb58700 1 -- 172.21.15.155:0/598477172 wait complete. 2024-03-21T12:51:45.514 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.510+0000 7fa99eb58700 1 Processor -- start 2024-03-21T12:51:45.514 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.510+0000 7fa99eb58700 1 -- start start 2024-03-21T12:51:45.514 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.510+0000 7fa99eb58700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa9981a47e0 con 0x7fa998101fa0 2024-03-21T12:51:45.514 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.510+0000 7fa99eb58700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa9981a5a20 con 0x7fa99810a110 2024-03-21T12:51:45.514 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.510+0000 7fa99eb58700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa9981a6c60 con 0x7fa9981063d0 2024-03-21T12:51:45.514 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.510+0000 7fa99d0f5700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fa99810a110 0x7fa9981a2ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:54010/0 (socket says 172.21.15.155:54010) 2024-03-21T12:51:45.514 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.510+0000 7fa99d0f5700 1 -- 172.21.15.155:0/2380735488 learned_addr learned my addr 172.21.15.155:0/2380735488 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-03-21T12:51:45.514 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.510+0000 7fa995ffb700 1 -- 172.21.15.155:0/2380735488 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1566490582 0 0) 0x7fa9981a5a20 con 0x7fa99810a110 2024-03-21T12:51:45.514 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.510+0000 7fa995ffb700 1 -- 172.21.15.155:0/2380735488 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9880035b0 con 0x7fa99810a110 2024-03-21T12:51:45.514 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.511+0000 7fa995ffb700 1 -- 172.21.15.155:0/2380735488 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2511379212 0 0) 0x7fa9981a47e0 con 0x7fa998101fa0 2024-03-21T12:51:45.514 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.511+0000 7fa995ffb700 1 -- 172.21.15.155:0/2380735488 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9981a5a20 con 0x7fa998101fa0 2024-03-21T12:51:45.515 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.511+0000 7fa995ffb700 1 -- 172.21.15.155:0/2380735488 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4155964480 0 0) 0x7fa9981a6c60 con 0x7fa9981063d0 2024-03-21T12:51:45.515 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.511+0000 7fa995ffb700 1 -- 172.21.15.155:0/2380735488 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9981a47e0 con 0x7fa9981063d0 2024-03-21T12:51:45.515 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.511+0000 7fa995ffb700 1 -- 172.21.15.155:0/2380735488 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1999970733 0 0) 0x7fa9880035b0 con 0x7fa99810a110 2024-03-21T12:51:45.515 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.511+0000 7fa995ffb700 1 -- 172.21.15.155:0/2380735488 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa9981a6c60 con 0x7fa99810a110 2024-03-21T12:51:45.515 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.511+0000 7fa995ffb700 1 -- 172.21.15.155:0/2380735488 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fa990001c10 con 0x7fa99810a110 2024-03-21T12:51:45.515 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.511+0000 7fa995ffb700 1 -- 172.21.15.155:0/2380735488 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2345830644 0 0) 0x7fa9981a6c60 con 0x7fa99810a110 2024-03-21T12:51:45.515 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.511+0000 7fa995ffb700 1 -- 172.21.15.155:0/2380735488 >> v1:172.21.15.43:6790/0 conn(0x7fa9981063d0 legacy=0x7fa99819f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:45.515 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.511+0000 7fa995ffb700 1 -- 172.21.15.155:0/2380735488 >> v1:172.21.15.43:6789/0 conn(0x7fa998101fa0 legacy=0x7fa998075e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:45.515 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.511+0000 7fa995ffb700 1 -- 172.21.15.155:0/2380735488 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9981a7ea0 con 0x7fa99810a110 2024-03-21T12:51:45.515 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.511+0000 7fa99eb58700 1 -- 172.21.15.155:0/2380735488 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9981a4a10 con 0x7fa99810a110 2024-03-21T12:51:45.515 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.511+0000 7fa995ffb700 1 -- 172.21.15.155:0/2380735488 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7fa990003eb0 con 0x7fa99810a110 2024-03-21T12:51:45.515 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.511+0000 7fa995ffb700 1 -- 172.21.15.155:0/2380735488 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fa990005fb0 con 0x7fa99810a110 2024-03-21T12:51:45.515 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.511+0000 7fa99eb58700 1 -- 172.21.15.155:0/2380735488 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9981a4f50 con 0x7fa99810a110 2024-03-21T12:51:45.515 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.512+0000 7fa99eb58700 1 -- 172.21.15.155:0/2380735488 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa998068a80 con 0x7fa99810a110 2024-03-21T12:51:45.516 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.513+0000 7fa995ffb700 1 -- 172.21.15.155:0/2380735488 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 15) v1 ==== 92095+0+0 (unknown 2546926431 0 0) 0x7fa990003a20 con 0x7fa99810a110 2024-03-21T12:51:45.517 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.513+0000 7fa995ffb700 1 -- 172.21.15.155:0/2380735488 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(69..69 src has 1..69) v4 ==== 5942+0+0 (unknown 772786965 0 0) 0x7fa990089a20 con 0x7fa99810a110 2024-03-21T12:51:45.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.517+0000 7fa995ffb700 1 -- 172.21.15.155:0/2380735488 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7fa990057170 con 0x7fa99810a110 2024-03-21T12:51:45.623 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:45 smithi155 ceph-mon[33846]: pgmap v205: 132 pgs: 89 active+clean, 1 creating+activating, 26 creating+peering, 16 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-21T12:51:45.624 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:45 smithi155 ceph-mon[33846]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-21T12:51:45.624 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:45 smithi155 ceph-mon[33846]: Cluster is now healthy 2024-03-21T12:51:45.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.728+0000 7fa99eb58700 1 -- 172.21.15.155:0/2380735488 --> v1:172.21.15.43:6800/4081968544 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi155=a", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9981a5380 con 0x7fa98806ece0 2024-03-21T12:51:45.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:45 smithi043 ceph-mon[30062]: pgmap v205: 132 pgs: 89 active+clean, 1 creating+activating, 26 creating+peering, 16 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-21T12:51:45.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:45 smithi043 ceph-mon[30062]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-21T12:51:45.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:45 smithi043 ceph-mon[30062]: Cluster is now healthy 2024-03-21T12:51:45.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:45 smithi043 ceph-mon[41699]: pgmap v205: 132 pgs: 89 active+clean, 1 creating+activating, 26 creating+peering, 16 unknown; 578 KiB data, 225 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-21T12:51:45.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:45 smithi043 ceph-mon[41699]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-21T12:51:45.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:45 smithi043 ceph-mon[41699]: Cluster is now healthy 2024-03-21T12:51:45.737 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.736+0000 7fa995ffb700 1 -- 172.21.15.155:0/2380735488 <== mgr.14152 v1:172.21.15.43:6800/4081968544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (unknown 0 0 1342662408) 0x7fa9981a5380 con 0x7fa98806ece0 2024-03-21T12:51:45.737 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled prometheus update... 2024-03-21T12:51:45.739 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.738+0000 7fa99eb58700 1 -- 172.21.15.155:0/2380735488 >> v1:172.21.15.43:6800/4081968544 conn(0x7fa98806ece0 legacy=0x7fa9880711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:45.739 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.738+0000 7fa99eb58700 1 -- 172.21.15.155:0/2380735488 >> v1:172.21.15.155:6789/0 conn(0x7fa99810a110 legacy=0x7fa9981a2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:45.739 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.738+0000 7fa99eb58700 1 -- 172.21.15.155:0/2380735488 shutdown_connections 2024-03-21T12:51:45.739 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.738+0000 7fa99eb58700 1 -- 172.21.15.155:0/2380735488 >> 172.21.15.155:0/2380735488 conn(0x7fa9980fd540 msgr2=0x7fa998108820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:51:45.739 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.738+0000 7fa99eb58700 1 -- 172.21.15.155:0/2380735488 shutdown_connections 2024-03-21T12:51:45.740 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:45.738+0000 7fa99eb58700 1 -- 172.21.15.155:0/2380735488 wait complete. 2024-03-21T12:51:46.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:51:46 smithi155 systemd[1]: Starting Ceph iscsi.iscsi.a for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T12:51:46.781 DEBUG:teuthology.orchestra.run.smithi155:prometheus.a> sudo journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@prometheus.a.service 2024-03-21T12:51:46.783 INFO:tasks.cephadm:Adding node-exporter.a on smithi043 2024-03-21T12:51:46.783 INFO:tasks.cephadm:Adding node-exporter.b on smithi155 2024-03-21T12:51:46.784 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph orch apply node-exporter '2;smithi043=a;smithi155=b' 2024-03-21T12:51:46.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:46 smithi155 ceph-mon[33846]: from='client.24433 v1:172.21.15.155:0/2380735488' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi155=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:51:46.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:46 smithi155 ceph-mon[33846]: Saving service prometheus spec with placement smithi155=a;count:1 2024-03-21T12:51:46.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:46 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:46.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:46 smithi155 ceph-mon[33846]: osdmap e70: 8 total, 8 up, 8 in 2024-03-21T12:51:47.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:46 smithi043 ceph-mon[41699]: from='client.24433 v1:172.21.15.155:0/2380735488' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi155=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:51:47.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:46 smithi043 ceph-mon[41699]: Saving service prometheus spec with placement smithi155=a;count:1 2024-03-21T12:51:47.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:46 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:47.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:46 smithi043 ceph-mon[41699]: osdmap e70: 8 total, 8 up, 8 in 2024-03-21T12:51:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:46 smithi043 ceph-mon[30062]: from='client.24433 v1:172.21.15.155:0/2380735488' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi155=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:51:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:46 smithi043 ceph-mon[30062]: Saving service prometheus spec with placement smithi155=a;count:1 2024-03-21T12:51:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:46 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:46 smithi043 ceph-mon[30062]: osdmap e70: 8 total, 8 up, 8 in 2024-03-21T12:51:47.250 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:51:46 smithi155 podman[63340]: 2024-03-21 12:51:46.897869129 +0000 UTC m=+0.019556118 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93 2024-03-21T12:51:47.250 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:51:47 smithi155 podman[63340]: 2024-03-21 12:51:47.01050588 +0000 UTC m=+0.132192849 container create f6ef43e8976ade7ff34199a453bcd937009acd06b7b927a09eb5c736aaf67533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a, GIT_CLEAN=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T12:51:47.250 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:51:47 smithi155 podman[63340]: 2024-03-21 12:51:47.250284654 +0000 UTC m=+0.371971636 container init f6ef43e8976ade7ff34199a453bcd937009acd06b7b927a09eb5c736aaf67533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-03-21T12:51:47.418 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T12:51:47.541 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:51:47 smithi155 podman[63340]: 2024-03-21 12:51:47.255703143 +0000 UTC m=+0.377390112 container start f6ef43e8976ade7ff34199a453bcd937009acd06b7b927a09eb5c736aaf67533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-21T12:51:47.541 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:51:47 smithi155 bash[63340]: f6ef43e8976ade7ff34199a453bcd937009acd06b7b927a09eb5c736aaf67533 2024-03-21T12:51:47.541 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:51:47 smithi155 systemd[1]: Started Ceph iscsi.iscsi.a for c06c8d1a-e780-11ee-95cd-87774f69a715. 2024-03-21T12:51:47.827 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:47 smithi155 ceph-mon[33846]: pgmap v206: 132 pgs: 120 active+clean, 1 creating+activating, 11 creating+peering; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 3.4 KiB/s wr, 102 op/s 2024-03-21T12:51:47.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:47 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:47.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:47 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:47.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:47 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:47.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:47 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:47.828 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:47 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:51:47.828 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:51:47 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug Started the configuration object watcher 2024-03-21T12:51:47.828 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:51:47 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug Checking for config object changes every 1s 2024-03-21T12:51:47.828 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:51:47 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug Processing osd blocklist entries for this node 2024-03-21T12:51:48.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:51:47 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug Reading the configuration object to update local LIO configuration 2024-03-21T12:51:48.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:51:47 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug Configuration does not have an entry for this host(smithi155.front.sepia.ceph.com) - nothing to define to LIO 2024-03-21T12:51:48.160 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:51:47 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug * Running on http://[::]:5000/ (Press CTRL+C to quit) 2024-03-21T12:51:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:47 smithi043 ceph-mon[30062]: pgmap v206: 132 pgs: 120 active+clean, 1 creating+activating, 11 creating+peering; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 3.4 KiB/s wr, 102 op/s 2024-03-21T12:51:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:47 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:47 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:47 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:47 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:47 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:51:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:47 smithi043 ceph-mon[41699]: pgmap v206: 132 pgs: 120 active+clean, 1 creating+activating, 11 creating+peering; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 3.4 KiB/s wr, 102 op/s 2024-03-21T12:51:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:47 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:47 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:47 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:47 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:47 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:51:49.077 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:48 smithi155 ceph-mon[33846]: Checking pool "datapool" exists for service iscsi.datapool 2024-03-21T12:51:49.077 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:48 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.155:0/124824410' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T12:51:49.077 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:48 smithi155 ceph-mon[33846]: mgrmap e16: y(active, since 4m), standbys: x 2024-03-21T12:51:49.090 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:48 smithi043 ceph-mon[30062]: Checking pool "datapool" exists for service iscsi.datapool 2024-03-21T12:51:49.090 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:48 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.155:0/124824410' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T12:51:49.090 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:48 smithi043 ceph-mon[30062]: mgrmap e16: y(active, since 4m), standbys: x 2024-03-21T12:51:49.090 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:48 smithi043 ceph-mon[41699]: Checking pool "datapool" exists for service iscsi.datapool 2024-03-21T12:51:49.090 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:48 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.155:0/124824410' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T12:51:49.090 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:48 smithi043 ceph-mon[41699]: mgrmap e16: y(active, since 4m), standbys: x 2024-03-21T12:51:49.724 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.723+0000 7f3e425fe700 1 -- 172.21.15.155:0/3875541714 >> v1:172.21.15.155:6789/0 conn(0x7f3e3c105330 legacy=0x7f3e3c107720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:49.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.723+0000 7f3e425fe700 1 -- 172.21.15.155:0/3875541714 shutdown_connections 2024-03-21T12:51:49.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.723+0000 7f3e425fe700 1 -- 172.21.15.155:0/3875541714 >> 172.21.15.155:0/3875541714 conn(0x7f3e3c0fc470 msgr2=0x7f3e3c0fe8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:51:49.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.723+0000 7f3e425fe700 1 -- 172.21.15.155:0/3875541714 shutdown_connections 2024-03-21T12:51:49.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.723+0000 7f3e425fe700 1 -- 172.21.15.155:0/3875541714 wait complete. 2024-03-21T12:51:49.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.724+0000 7f3e425fe700 1 Processor -- start 2024-03-21T12:51:49.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.724+0000 7f3e425fe700 1 -- start start 2024-03-21T12:51:49.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.724+0000 7f3e425fe700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3e3c06b3d0 con 0x7f3e3c105330 2024-03-21T12:51:49.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.724+0000 7f3e425fe700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3e3c1a3880 con 0x7f3e3c108f60 2024-03-21T12:51:49.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.724+0000 7f3e425fe700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3e3c1a4aa0 con 0x7f3e3c100e80 2024-03-21T12:51:49.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.725+0000 7f3e40b9b700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f3e3c108f60 0x7f3e3c1a1150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:54102/0 (socket says 172.21.15.155:54102) 2024-03-21T12:51:49.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.725+0000 7f3e40b9b700 1 -- 172.21.15.155:0/4195387159 learned_addr learned my addr 172.21.15.155:0/4195387159 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-03-21T12:51:49.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.725+0000 7f3e397fa700 1 -- 172.21.15.155:0/4195387159 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3839537301 0 0) 0x7f3e3c06b3d0 con 0x7f3e3c105330 2024-03-21T12:51:49.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.725+0000 7f3e397fa700 1 -- 172.21.15.155:0/4195387159 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e240035b0 con 0x7f3e3c105330 2024-03-21T12:51:49.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.725+0000 7f3e397fa700 1 -- 172.21.15.155:0/4195387159 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1321595422 0 0) 0x7f3e3c1a3880 con 0x7f3e3c108f60 2024-03-21T12:51:49.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.725+0000 7f3e397fa700 1 -- 172.21.15.155:0/4195387159 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e3c06b3d0 con 0x7f3e3c108f60 2024-03-21T12:51:49.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.725+0000 7f3e397fa700 1 -- 172.21.15.155:0/4195387159 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3744223775 0 0) 0x7f3e3c1a4aa0 con 0x7f3e3c100e80 2024-03-21T12:51:49.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.725+0000 7f3e397fa700 1 -- 172.21.15.155:0/4195387159 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e3c1a3880 con 0x7f3e3c100e80 2024-03-21T12:51:49.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.725+0000 7f3e397fa700 1 -- 172.21.15.155:0/4195387159 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1750235775 0 0) 0x7f3e240035b0 con 0x7f3e3c105330 2024-03-21T12:51:49.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.725+0000 7f3e397fa700 1 -- 172.21.15.155:0/4195387159 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3e3c1a4aa0 con 0x7f3e3c105330 2024-03-21T12:51:49.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.725+0000 7f3e397fa700 1 -- 172.21.15.155:0/4195387159 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f3e30001430 con 0x7f3e3c105330 2024-03-21T12:51:49.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.725+0000 7f3e397fa700 1 -- 172.21.15.155:0/4195387159 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1630562500 0 0) 0x7f3e3c1a3880 con 0x7f3e3c100e80 2024-03-21T12:51:49.728 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.725+0000 7f3e397fa700 1 -- 172.21.15.155:0/4195387159 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3e240035b0 con 0x7f3e3c100e80 2024-03-21T12:51:49.728 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.725+0000 7f3e397fa700 1 -- 172.21.15.155:0/4195387159 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1813186882 0 0) 0x7f3e3c06b3d0 con 0x7f3e3c108f60 2024-03-21T12:51:49.728 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.725+0000 7f3e397fa700 1 -- 172.21.15.155:0/4195387159 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3e3c1a3880 con 0x7f3e3c108f60 2024-03-21T12:51:49.728 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.725+0000 7f3e397fa700 1 -- 172.21.15.155:0/4195387159 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f3e2c001770 con 0x7f3e3c100e80 2024-03-21T12:51:49.728 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.725+0000 7f3e397fa700 1 -- 172.21.15.155:0/4195387159 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f3e34001b50 con 0x7f3e3c108f60 2024-03-21T12:51:49.728 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.725+0000 7f3e397fa700 1 -- 172.21.15.155:0/4195387159 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2454996492 0 0) 0x7f3e3c1a4aa0 con 0x7f3e3c105330 2024-03-21T12:51:49.728 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.725+0000 7f3e397fa700 1 -- 172.21.15.155:0/4195387159 >> v1:172.21.15.43:6790/0 conn(0x7f3e3c100e80 legacy=0x7f3e3c19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:49.728 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.725+0000 7f3e397fa700 1 -- 172.21.15.155:0/4195387159 >> v1:172.21.15.155:6789/0 conn(0x7f3e3c108f60 legacy=0x7f3e3c1a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:49.728 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.725+0000 7f3e397fa700 1 -- 172.21.15.155:0/4195387159 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e3c1a5cc0 con 0x7f3e3c105330 2024-03-21T12:51:49.728 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.726+0000 7f3e425fe700 1 -- 172.21.15.155:0/4195387159 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e3c1a3ab0 con 0x7f3e3c105330 2024-03-21T12:51:49.728 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.726+0000 7f3e425fe700 1 -- 172.21.15.155:0/4195387159 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e3c1a3ff0 con 0x7f3e3c105330 2024-03-21T12:51:49.728 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.726+0000 7f3e397fa700 1 -- 172.21.15.155:0/4195387159 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f3e30001690 con 0x7f3e3c105330 2024-03-21T12:51:49.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.726+0000 7f3e397fa700 1 -- 172.21.15.155:0/4195387159 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f3e30004d10 con 0x7f3e3c105330 2024-03-21T12:51:49.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.727+0000 7f3e425fe700 1 -- 172.21.15.155:0/4195387159 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e28005150 con 0x7f3e3c105330 2024-03-21T12:51:49.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.728+0000 7f3e397fa700 1 -- 172.21.15.155:0/4195387159 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 16) v1 ==== 92146+0+0 (unknown 510893937 0 0) 0x7f3e3001b510 con 0x7f3e3c105330 2024-03-21T12:51:49.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.728+0000 7f3e397fa700 1 -- 172.21.15.155:0/4195387159 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(70..70 src has 1..70) v4 ==== 5942+0+0 (unknown 1984516339 0 0) 0x7f3e30088490 con 0x7f3e3c105330 2024-03-21T12:51:49.734 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.733+0000 7f3e397fa700 1 -- 172.21.15.155:0/4195387159 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f3e30055b00 con 0x7f3e3c105330 2024-03-21T12:51:49.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:49 smithi155 ceph-mon[33846]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s rd, 5.8 KiB/s wr, 187 op/s 2024-03-21T12:51:49.968 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.967+0000 7f3e425fe700 1 -- 172.21.15.155:0/4195387159 --> v1:172.21.15.43:6800/4081968544 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi043=a;smithi155=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e28000bc0 con 0x7f3e24077a30 2024-03-21T12:51:49.975 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.974+0000 7f3e397fa700 1 -- 172.21.15.155:0/4195387159 <== mgr.14152 v1:172.21.15.43:6800/4081968544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (unknown 0 0 240551134) 0x7f3e28000bc0 con 0x7f3e24077a30 2024-03-21T12:51:49.975 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled node-exporter update... 2024-03-21T12:51:49.977 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.977+0000 7f3e425fe700 1 -- 172.21.15.155:0/4195387159 >> v1:172.21.15.43:6800/4081968544 conn(0x7f3e24077a30 legacy=0x7f3e24079ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:49.977 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.977+0000 7f3e425fe700 1 -- 172.21.15.155:0/4195387159 >> v1:172.21.15.43:6789/0 conn(0x7f3e3c105330 legacy=0x7f3e3c19da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:49.978 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.977+0000 7f3e425fe700 1 -- 172.21.15.155:0/4195387159 shutdown_connections 2024-03-21T12:51:49.978 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.977+0000 7f3e425fe700 1 -- 172.21.15.155:0/4195387159 >> 172.21.15.155:0/4195387159 conn(0x7f3e3c0fc470 msgr2=0x7f3e3c10b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:51:49.978 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.977+0000 7f3e425fe700 1 -- 172.21.15.155:0/4195387159 shutdown_connections 2024-03-21T12:51:49.978 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:49.977+0000 7f3e425fe700 1 -- 172.21.15.155:0/4195387159 wait complete. 2024-03-21T12:51:49.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:49 smithi043 ceph-mon[30062]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s rd, 5.8 KiB/s wr, 187 op/s 2024-03-21T12:51:49.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:49 smithi043 ceph-mon[41699]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 77 KiB/s rd, 5.8 KiB/s wr, 187 op/s 2024-03-21T12:51:50.745 DEBUG:teuthology.orchestra.run.smithi043:node-exporter.a> sudo journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@node-exporter.a.service 2024-03-21T12:51:50.747 DEBUG:teuthology.orchestra.run.smithi155:node-exporter.b> sudo journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@node-exporter.b.service 2024-03-21T12:51:50.750 INFO:tasks.cephadm:Adding alertmanager.a on smithi043 2024-03-21T12:51:50.750 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph orch apply alertmanager '1;smithi043=a' 2024-03-21T12:51:51.209 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T12:51:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:50 smithi043 ceph-mon[30062]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 4.4 KiB/s wr, 142 op/s 2024-03-21T12:51:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:50 smithi043 ceph-mon[30062]: from='client.14565 v1:172.21.15.155:0/4195387159' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi043=a;smithi155=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:51:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:50 smithi043 ceph-mon[30062]: Saving service node-exporter spec with placement smithi043=a;smithi155=b;count:2 2024-03-21T12:51:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:50 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:50 smithi043 ceph-mon[41699]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 4.4 KiB/s wr, 142 op/s 2024-03-21T12:51:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:50 smithi043 ceph-mon[41699]: from='client.14565 v1:172.21.15.155:0/4195387159' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi043=a;smithi155=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:51:51.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:50 smithi043 ceph-mon[41699]: Saving service node-exporter spec with placement smithi043=a;smithi155=b;count:2 2024-03-21T12:51:51.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:50 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:51.336 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:50 smithi155 ceph-mon[33846]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 59 KiB/s rd, 4.4 KiB/s wr, 142 op/s 2024-03-21T12:51:51.337 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:50 smithi155 ceph-mon[33846]: from='client.14565 v1:172.21.15.155:0/4195387159' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi043=a;smithi155=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:51:51.337 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:50 smithi155 ceph-mon[33846]: Saving service node-exporter spec with placement smithi043=a;smithi155=b;count:2 2024-03-21T12:51:51.337 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:50 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:52.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:51 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-03-21T12:51:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:51 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2024-03-21T12:51:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:51 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2024-03-21T12:51:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:51 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-03-21T12:51:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:51 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2024-03-21T12:51:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:51 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2024-03-21T12:51:52.359 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:51 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-03-21T12:51:52.359 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:51 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]: dispatch 2024-03-21T12:51:52.359 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:51 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]: dispatch 2024-03-21T12:51:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:52 smithi043 ceph-mon[30062]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 53 KiB/s rd, 4.1 KiB/s wr, 128 op/s 2024-03-21T12:51:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:52 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-03-21T12:51:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:52 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]': finished 2024-03-21T12:51:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:52 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-03-21T12:51:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:52 smithi043 ceph-mon[30062]: osdmap e71: 8 total, 8 up, 8 in 2024-03-21T12:51:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:52 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:53.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:52 smithi043 ceph-mon[41699]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 53 KiB/s rd, 4.1 KiB/s wr, 128 op/s 2024-03-21T12:51:53.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:52 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-03-21T12:51:53.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:52 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]': finished 2024-03-21T12:51:53.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:52 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-03-21T12:51:53.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:52 smithi043 ceph-mon[41699]: osdmap e71: 8 total, 8 up, 8 in 2024-03-21T12:51:53.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:52 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:53.235 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:52 smithi155 ceph-mon[33846]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 53 KiB/s rd, 4.1 KiB/s wr, 128 op/s 2024-03-21T12:51:53.235 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:52 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-03-21T12:51:53.235 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:52 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [1, 2]}]': finished 2024-03-21T12:51:53.235 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:52 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [1, 5]}]': finished 2024-03-21T12:51:53.235 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:52 smithi155 ceph-mon[33846]: osdmap e71: 8 total, 8 up, 8 in 2024-03-21T12:51:53.235 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:52 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:53.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.233+0000 7f206bfbe700 1 -- 172.21.15.155:0/1645438733 >> v1:172.21.15.155:6789/0 conn(0x7f2064100630 legacy=0x7f2064100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:53.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.233+0000 7f206bfbe700 1 -- 172.21.15.155:0/1645438733 shutdown_connections 2024-03-21T12:51:53.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.233+0000 7f206bfbe700 1 -- 172.21.15.155:0/1645438733 >> 172.21.15.155:0/1645438733 conn(0x7f20640fbbd0 msgr2=0x7f20640fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:51:53.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.234+0000 7f206bfbe700 1 -- 172.21.15.155:0/1645438733 shutdown_connections 2024-03-21T12:51:53.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.234+0000 7f206bfbe700 1 -- 172.21.15.155:0/1645438733 wait complete. 2024-03-21T12:51:53.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.234+0000 7f206bfbe700 1 Processor -- start 2024-03-21T12:51:53.266 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.234+0000 7f206bfbe700 1 -- start start 2024-03-21T12:51:53.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.234+0000 7f206bfbe700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f20641a43e0 con 0x7f20641087a0 2024-03-21T12:51:53.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.234+0000 7f206bfbe700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f20641a5620 con 0x7f2064100630 2024-03-21T12:51:53.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.234+0000 7f206bfbe700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f20641a6860 con 0x7f2064104a60 2024-03-21T12:51:53.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.235+0000 7f2069d5a700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f2064100630 0x7f2064106930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:33758/0 (socket says 172.21.15.155:33758) 2024-03-21T12:51:53.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.235+0000 7f2069d5a700 1 -- 172.21.15.155:0/1575901434 learned_addr learned my addr 172.21.15.155:0/1575901434 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-03-21T12:51:53.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.235+0000 7f205affd700 1 -- 172.21.15.155:0/1575901434 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2503969574 0 0) 0x7f20641a5620 con 0x7f2064100630 2024-03-21T12:51:53.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.235+0000 7f205affd700 1 -- 172.21.15.155:0/1575901434 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20500046c0 con 0x7f2064100630 2024-03-21T12:51:53.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.235+0000 7f205affd700 1 -- 172.21.15.155:0/1575901434 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3064975875 0 0) 0x7f20641a6860 con 0x7f2064104a60 2024-03-21T12:51:53.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.235+0000 7f205affd700 1 -- 172.21.15.155:0/1575901434 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20641a5620 con 0x7f2064104a60 2024-03-21T12:51:53.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.235+0000 7f205affd700 1 -- 172.21.15.155:0/1575901434 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4162445256 0 0) 0x7f20500046c0 con 0x7f2064100630 2024-03-21T12:51:53.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.235+0000 7f205affd700 1 -- 172.21.15.155:0/1575901434 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f20641a6860 con 0x7f2064100630 2024-03-21T12:51:53.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.235+0000 7f205affd700 1 -- 172.21.15.155:0/1575901434 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f2060004740 con 0x7f2064100630 2024-03-21T12:51:53.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.235+0000 7f205affd700 1 -- 172.21.15.155:0/1575901434 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 65143933 0 0) 0x7f20641a43e0 con 0x7f20641087a0 2024-03-21T12:51:53.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.235+0000 7f205affd700 1 -- 172.21.15.155:0/1575901434 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20500046c0 con 0x7f20641087a0 2024-03-21T12:51:53.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.235+0000 7f205affd700 1 -- 172.21.15.155:0/1575901434 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3216174979 0 0) 0x7f20641a6860 con 0x7f2064100630 2024-03-21T12:51:53.267 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.235+0000 7f205affd700 1 -- 172.21.15.155:0/1575901434 >> v1:172.21.15.43:6790/0 conn(0x7f2064104a60 legacy=0x7f206419f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:53.268 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.235+0000 7f205affd700 1 -- 172.21.15.155:0/1575901434 >> v1:172.21.15.43:6789/0 conn(0x7f20641087a0 legacy=0x7f20641a2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:53.268 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.235+0000 7f205affd700 1 -- 172.21.15.155:0/1575901434 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20641a7aa0 con 0x7f2064100630 2024-03-21T12:51:53.268 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.235+0000 7f206bfbe700 1 -- 172.21.15.155:0/1575901434 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20641a6a90 con 0x7f2064100630 2024-03-21T12:51:53.268 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.235+0000 7f206bfbe700 1 -- 172.21.15.155:0/1575901434 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f20641a6f50 con 0x7f2064100630 2024-03-21T12:51:53.268 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.235+0000 7f205affd700 1 -- 172.21.15.155:0/1575901434 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f2060003700 con 0x7f2064100630 2024-03-21T12:51:53.268 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.235+0000 7f205affd700 1 -- 172.21.15.155:0/1575901434 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f2060005080 con 0x7f2064100630 2024-03-21T12:51:53.268 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.236+0000 7f206bfbe700 1 -- 172.21.15.155:0/1575901434 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2048005120 con 0x7f2064100630 2024-03-21T12:51:53.268 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.237+0000 7f205affd700 1 -- 172.21.15.155:0/1575901434 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 16) v1 ==== 92146+0+0 (unknown 510893937 0 0) 0x7f2060005300 con 0x7f2064100630 2024-03-21T12:51:53.268 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.237+0000 7f205affd700 1 -- 172.21.15.155:0/1575901434 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(72..72 src has 1..72) v4 ==== 6001+0+0 (unknown 82262620 0 0) 0x7f2060089d00 con 0x7f2064100630 2024-03-21T12:51:53.268 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.242+0000 7f205affd700 1 -- 172.21.15.155:0/1575901434 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f20600572b0 con 0x7f2064100630 2024-03-21T12:51:53.457 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.457+0000 7f206bfbe700 1 -- 172.21.15.155:0/1575901434 --> v1:172.21.15.43:6800/4081968544 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi043=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f2048000b90 con 0x7f205006f160 2024-03-21T12:51:53.464 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.463+0000 7f205affd700 1 -- 172.21.15.155:0/1575901434 <== mgr.14152 v1:172.21.15.43:6800/4081968544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (unknown 0 0 1850065467) 0x7f2048000b90 con 0x7f205006f160 2024-03-21T12:51:53.464 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled alertmanager update... 2024-03-21T12:51:53.466 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.465+0000 7f206bfbe700 1 -- 172.21.15.155:0/1575901434 >> v1:172.21.15.43:6800/4081968544 conn(0x7f205006f160 legacy=0x7f2050071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:53.466 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.466+0000 7f206bfbe700 1 -- 172.21.15.155:0/1575901434 >> v1:172.21.15.155:6789/0 conn(0x7f2064100630 legacy=0x7f2064106930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:53.467 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.466+0000 7f206bfbe700 1 -- 172.21.15.155:0/1575901434 shutdown_connections 2024-03-21T12:51:53.467 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.466+0000 7f206bfbe700 1 -- 172.21.15.155:0/1575901434 >> 172.21.15.155:0/1575901434 conn(0x7f20640fbbd0 msgr2=0x7f20641075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:51:53.467 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.466+0000 7f206bfbe700 1 -- 172.21.15.155:0/1575901434 shutdown_connections 2024-03-21T12:51:53.467 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:53.466+0000 7f206bfbe700 1 -- 172.21.15.155:0/1575901434 wait complete. 2024-03-21T12:51:54.113 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:53 smithi155 ceph-mon[33846]: osdmap e72: 8 total, 8 up, 8 in 2024-03-21T12:51:54.114 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:53 smithi155 ceph-mon[33846]: from='client.24475 v1:172.21.15.155:0/1575901434' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi043=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:51:54.114 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:53 smithi155 ceph-mon[33846]: Saving service alertmanager spec with placement smithi043=a;count:1 2024-03-21T12:51:54.114 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:53 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:54.114 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:53 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:54.114 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:53 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:54.114 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:53 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:54.114 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:53 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:51:54.114 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:53 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:54.114 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:53 smithi155 ceph-mon[33846]: Deploying daemon prometheus.a on smithi155 2024-03-21T12:51:54.140 DEBUG:teuthology.orchestra.run.smithi043:alertmanager.a> sudo journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@alertmanager.a.service 2024-03-21T12:51:54.142 INFO:tasks.cephadm:Adding grafana.a on smithi155 2024-03-21T12:51:54.142 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph orch apply grafana '1;smithi155=a' 2024-03-21T12:51:54.391 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T12:51:54.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:53 smithi043 ceph-mon[30062]: osdmap e72: 8 total, 8 up, 8 in 2024-03-21T12:51:54.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:53 smithi043 ceph-mon[30062]: from='client.24475 v1:172.21.15.155:0/1575901434' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi043=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:51:54.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:53 smithi043 ceph-mon[30062]: Saving service alertmanager spec with placement smithi043=a;count:1 2024-03-21T12:51:54.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:53 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:54.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:53 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:54.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:53 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:54.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:53 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:54.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:53 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:51:54.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:53 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:54.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:53 smithi043 ceph-mon[30062]: Deploying daemon prometheus.a on smithi155 2024-03-21T12:51:54.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:53 smithi043 ceph-mon[41699]: osdmap e72: 8 total, 8 up, 8 in 2024-03-21T12:51:54.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:53 smithi043 ceph-mon[41699]: from='client.24475 v1:172.21.15.155:0/1575901434' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi043=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:51:54.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:53 smithi043 ceph-mon[41699]: Saving service alertmanager spec with placement smithi043=a;count:1 2024-03-21T12:51:54.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:53 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:54.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:53 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:54.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:53 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:54.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:53 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:51:54.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:53 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:51:54.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:53 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:54.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:53 smithi043 ceph-mon[41699]: Deploying daemon prometheus.a on smithi155 2024-03-21T12:51:55.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:55 smithi155 ceph-mon[33846]: pgmap v213: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 1.4 KiB/s wr, 45 op/s 2024-03-21T12:51:55.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:55 smithi043 ceph-mon[30062]: pgmap v213: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 1.4 KiB/s wr, 45 op/s 2024-03-21T12:51:55.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:55 smithi043 ceph-mon[41699]: pgmap v213: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 1.4 KiB/s wr, 45 op/s 2024-03-21T12:51:56.751 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.716+0000 7f198eb9e700 1 -- 172.21.15.155:0/1787982529 >> v1:172.21.15.155:6789/0 conn(0x7f1988106210 legacy=0x7f1988108660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:56.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.716+0000 7f198eb9e700 1 -- 172.21.15.155:0/1787982529 shutdown_connections 2024-03-21T12:51:56.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.716+0000 7f198eb9e700 1 -- 172.21.15.155:0/1787982529 >> 172.21.15.155:0/1787982529 conn(0x7f19880fd360 msgr2=0x7f19880ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:51:56.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.716+0000 7f198eb9e700 1 -- 172.21.15.155:0/1787982529 shutdown_connections 2024-03-21T12:51:56.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.716+0000 7f198eb9e700 1 -- 172.21.15.155:0/1787982529 wait complete. 2024-03-21T12:51:56.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.717+0000 7f198eb9e700 1 Processor -- start 2024-03-21T12:51:56.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.717+0000 7f198eb9e700 1 -- start start 2024-03-21T12:51:56.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.717+0000 7f198eb9e700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1988197b60 con 0x7f1988106210 2024-03-21T12:51:56.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.717+0000 7f198eb9e700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f19881a3350 con 0x7f1988101de0 2024-03-21T12:51:56.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.717+0000 7f198eb9e700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f19881a4530 con 0x7f1988109f50 2024-03-21T12:51:56.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.717+0000 7f198db9c700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f1988101de0 0x7f1988197010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:33772/0 (socket says 172.21.15.155:33772) 2024-03-21T12:51:56.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.717+0000 7f198db9c700 1 -- 172.21.15.155:0/3516116592 learned_addr learned my addr 172.21.15.155:0/3516116592 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-03-21T12:51:56.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.717+0000 7f197effd700 1 -- 172.21.15.155:0/3516116592 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1371300674 0 0) 0x7f19881a3350 con 0x7f1988101de0 2024-03-21T12:51:56.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.717+0000 7f197effd700 1 -- 172.21.15.155:0/3516116592 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19740035b0 con 0x7f1988101de0 2024-03-21T12:51:56.753 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.717+0000 7f197effd700 1 -- 172.21.15.155:0/3516116592 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1740484269 0 0) 0x7f1988197b60 con 0x7f1988106210 2024-03-21T12:51:56.753 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.717+0000 7f197effd700 1 -- 172.21.15.155:0/3516116592 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19881a3350 con 0x7f1988106210 2024-03-21T12:51:56.753 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.717+0000 7f197effd700 1 -- 172.21.15.155:0/3516116592 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 217133320 0 0) 0x7f19881a4530 con 0x7f1988109f50 2024-03-21T12:51:56.753 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.717+0000 7f197effd700 1 -- 172.21.15.155:0/3516116592 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1988197b60 con 0x7f1988109f50 2024-03-21T12:51:56.753 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.717+0000 7f197effd700 1 -- 172.21.15.155:0/3516116592 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3374288944 0 0) 0x7f19740035b0 con 0x7f1988101de0 2024-03-21T12:51:56.753 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.717+0000 7f197effd700 1 -- 172.21.15.155:0/3516116592 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f19881a4530 con 0x7f1988101de0 2024-03-21T12:51:56.753 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.717+0000 7f197effd700 1 -- 172.21.15.155:0/3516116592 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f1980001c70 con 0x7f1988101de0 2024-03-21T12:51:56.753 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.718+0000 7f197effd700 1 -- 172.21.15.155:0/3516116592 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2904215603 0 0) 0x7f19881a4530 con 0x7f1988101de0 2024-03-21T12:51:56.753 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.718+0000 7f197effd700 1 -- 172.21.15.155:0/3516116592 >> v1:172.21.15.43:6790/0 conn(0x7f1988109f50 legacy=0x7f19881a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:56.753 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.718+0000 7f197effd700 1 -- 172.21.15.155:0/3516116592 >> v1:172.21.15.43:6789/0 conn(0x7f1988106210 legacy=0x7f198819d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:56.753 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.718+0000 7f197effd700 1 -- 172.21.15.155:0/3516116592 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19881a5750 con 0x7f1988101de0 2024-03-21T12:51:56.753 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.718+0000 7f198eb9e700 1 -- 172.21.15.155:0/3516116592 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19881a23a0 con 0x7f1988101de0 2024-03-21T12:51:56.753 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.718+0000 7f198eb9e700 1 -- 172.21.15.155:0/3516116592 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f19881a28e0 con 0x7f1988101de0 2024-03-21T12:51:56.754 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.718+0000 7f197effd700 1 -- 172.21.15.155:0/3516116592 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f1980003eb0 con 0x7f1988101de0 2024-03-21T12:51:56.754 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.718+0000 7f197effd700 1 -- 172.21.15.155:0/3516116592 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f1980006070 con 0x7f1988101de0 2024-03-21T12:51:56.754 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.719+0000 7f198eb9e700 1 -- 172.21.15.155:0/3516116592 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f196c005150 con 0x7f1988101de0 2024-03-21T12:51:56.754 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.719+0000 7f197effd700 1 -- 172.21.15.155:0/3516116592 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 16) v1 ==== 92146+0+0 (unknown 510893937 0 0) 0x7f19800072e0 con 0x7f1988101de0 2024-03-21T12:51:56.754 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.720+0000 7f197effd700 1 -- 172.21.15.155:0/3516116592 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(72..72 src has 1..72) v4 ==== 6001+0+0 (unknown 82262620 0 0) 0x7f19800899d0 con 0x7f1988101de0 2024-03-21T12:51:56.754 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.724+0000 7f197effd700 1 -- 172.21.15.155:0/3516116592 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f1980057000 con 0x7f1988101de0 2024-03-21T12:51:56.934 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.933+0000 7f198eb9e700 1 -- 172.21.15.155:0/3516116592 --> v1:172.21.15.43:6800/4081968544 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi155=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f196c000bc0 con 0x7f197406ebc0 2024-03-21T12:51:56.941 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.940+0000 7f197effd700 1 -- 172.21.15.155:0/3516116592 <== mgr.14152 v1:172.21.15.43:6800/4081968544 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (unknown 0 0 664801700) 0x7f196c000bc0 con 0x7f197406ebc0 2024-03-21T12:51:56.941 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled grafana update... 2024-03-21T12:51:56.943 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.942+0000 7f198eb9e700 1 -- 172.21.15.155:0/3516116592 >> v1:172.21.15.43:6800/4081968544 conn(0x7f197406ebc0 legacy=0x7f1974071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:56.943 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.942+0000 7f198eb9e700 1 -- 172.21.15.155:0/3516116592 >> v1:172.21.15.155:6789/0 conn(0x7f1988101de0 legacy=0x7f1988197010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:51:56.943 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.942+0000 7f198eb9e700 1 -- 172.21.15.155:0/3516116592 shutdown_connections 2024-03-21T12:51:56.943 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.942+0000 7f198eb9e700 1 -- 172.21.15.155:0/3516116592 >> 172.21.15.155:0/3516116592 conn(0x7f19880fd360 msgr2=0x7f1988109650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:51:56.943 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.942+0000 7f198eb9e700 1 -- 172.21.15.155:0/3516116592 shutdown_connections 2024-03-21T12:51:56.943 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:51:56.942+0000 7f198eb9e700 1 -- 172.21.15.155:0/3516116592 wait complete. 2024-03-21T12:51:57.948 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:57 smithi155 ceph-mon[33846]: pgmap v214: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 127 B/s wr, 1 op/s 2024-03-21T12:51:57.948 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:57 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:57.948 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:57 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:57.948 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:51:57 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:51:57.977 DEBUG:teuthology.orchestra.run.smithi155:grafana.a> sudo journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@grafana.a.service 2024-03-21T12:51:57.980 INFO:tasks.cephadm:Setting up client nodes... 2024-03-21T12:51:57.980 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-21T12:51:57.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:57 smithi043 ceph-mon[30062]: pgmap v214: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 127 B/s wr, 1 op/s 2024-03-21T12:51:57.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:57 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:57.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:57 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:57.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:57 smithi043 ceph-mon[41699]: pgmap v214: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 127 B/s wr, 1 op/s 2024-03-21T12:51:57.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:57 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:57.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:57 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:51:58.235 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:51:58.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:58 smithi155 ceph-mon[33846]: from='client.24481 v1:172.21.15.155:0/3516116592' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi155=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:51:58.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:58 smithi155 ceph-mon[33846]: Saving service grafana spec with placement smithi155=a;count:1 2024-03-21T12:51:58.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:58 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:51:58.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:58 smithi043 ceph-mon[30062]: from='client.24481 v1:172.21.15.155:0/3516116592' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi155=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:51:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:58 smithi043 ceph-mon[30062]: Saving service grafana spec with placement smithi155=a;count:1 2024-03-21T12:51:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:58 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:51:58.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:58 smithi043 ceph-mon[41699]: from='client.24481 v1:172.21.15.155:0/3516116592' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi155=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T12:51:58.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:58 smithi043 ceph-mon[41699]: Saving service grafana spec with placement smithi155=a;count:1 2024-03-21T12:51:58.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:58 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:51:59.948 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:51:59 smithi043 ceph-mon[30062]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 127 B/s wr, 2 op/s; 41 B/s, 2 objects/s recovering 2024-03-21T12:51:59.948 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:51:59 smithi043 ceph-mon[41699]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 127 B/s wr, 2 op/s; 41 B/s, 2 objects/s recovering 2024-03-21T12:51:59.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:51:59 smithi155 ceph-mon[33846]: pgmap v215: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 127 B/s wr, 2 op/s; 41 B/s, 2 objects/s recovering 2024-03-21T12:52:00.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.325+0000 7f3251748700 1 -- 172.21.15.43:0/436328139 >> v1:172.21.15.43:6790/0 conn(0x7f324c10a100 legacy=0x7f324c10c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.325+0000 7f3251748700 1 -- 172.21.15.43:0/436328139 shutdown_connections 2024-03-21T12:52:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.325+0000 7f3251748700 1 -- 172.21.15.43:0/436328139 >> 172.21.15.43:0/436328139 conn(0x7f324c0fd530 msgr2=0x7f324c0ff970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.325+0000 7f3251748700 1 -- 172.21.15.43:0/436328139 shutdown_connections 2024-03-21T12:52:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.325+0000 7f3251748700 1 -- 172.21.15.43:0/436328139 wait complete. 2024-03-21T12:52:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.326+0000 7f3251748700 1 Processor -- start 2024-03-21T12:52:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.326+0000 7f3251748700 1 -- start start 2024-03-21T12:52:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.326+0000 7f3251748700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f324c197f00 con 0x7f324c10a100 2024-03-21T12:52:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.326+0000 7f3251748700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f324c1a36f0 con 0x7f324c101f90 2024-03-21T12:52:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.326+0000 7f3251748700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f324c1a48d0 con 0x7f324c1063c0 2024-03-21T12:52:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.326+0000 7f324b7fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f324c10a100 0x7f324c1a0fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38030/0 (socket says 172.21.15.43:38030) 2024-03-21T12:52:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.326+0000 7f324b7fe700 1 -- 172.21.15.43:0/2783673375 learned_addr learned my addr 172.21.15.43:0/2783673375 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:52:00.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.326+0000 7f3233fff700 1 -- 172.21.15.43:0/2783673375 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1509824182 0 0) 0x7f324c1a48d0 con 0x7f324c1063c0 2024-03-21T12:52:00.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.327+0000 7f3233fff700 1 -- 172.21.15.43:0/2783673375 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32340035b0 con 0x7f324c1063c0 2024-03-21T12:52:00.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.327+0000 7f3233fff700 1 -- 172.21.15.43:0/2783673375 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2447742322 0 0) 0x7f324c197f00 con 0x7f324c10a100 2024-03-21T12:52:00.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.327+0000 7f3233fff700 1 -- 172.21.15.43:0/2783673375 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f324c1a48d0 con 0x7f324c10a100 2024-03-21T12:52:00.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.327+0000 7f3233fff700 1 -- 172.21.15.43:0/2783673375 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637805448 0 0) 0x7f324c1a36f0 con 0x7f324c101f90 2024-03-21T12:52:00.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.327+0000 7f3233fff700 1 -- 172.21.15.43:0/2783673375 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f324c197f00 con 0x7f324c101f90 2024-03-21T12:52:00.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.327+0000 7f3233fff700 1 -- 172.21.15.43:0/2783673375 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1825542483 0 0) 0x7f324c1a48d0 con 0x7f324c10a100 2024-03-21T12:52:00.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.327+0000 7f3233fff700 1 -- 172.21.15.43:0/2783673375 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f324c1a36f0 con 0x7f324c10a100 2024-03-21T12:52:00.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.327+0000 7f3233fff700 1 -- 172.21.15.43:0/2783673375 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f3244004880 con 0x7f324c10a100 2024-03-21T12:52:00.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.327+0000 7f3233fff700 1 -- 172.21.15.43:0/2783673375 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1004721898 0 0) 0x7f324c197f00 con 0x7f324c101f90 2024-03-21T12:52:00.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.327+0000 7f3233fff700 1 -- 172.21.15.43:0/2783673375 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f324c1a48d0 con 0x7f324c101f90 2024-03-21T12:52:00.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.327+0000 7f3233fff700 1 -- 172.21.15.43:0/2783673375 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2062853599 0 0) 0x7f32340035b0 con 0x7f324c1063c0 2024-03-21T12:52:00.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.327+0000 7f3233fff700 1 -- 172.21.15.43:0/2783673375 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f324c197f00 con 0x7f324c1063c0 2024-03-21T12:52:00.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.327+0000 7f3233fff700 1 -- 172.21.15.43:0/2783673375 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f323c001130 con 0x7f324c101f90 2024-03-21T12:52:00.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.327+0000 7f3233fff700 1 -- 172.21.15.43:0/2783673375 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f3240001830 con 0x7f324c1063c0 2024-03-21T12:52:00.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.327+0000 7f3233fff700 1 -- 172.21.15.43:0/2783673375 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3305881882 0 0) 0x7f324c1a36f0 con 0x7f324c10a100 2024-03-21T12:52:00.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.327+0000 7f3233fff700 1 -- 172.21.15.43:0/2783673375 >> v1:172.21.15.43:6790/0 conn(0x7f324c1063c0 legacy=0x7f324c19d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:00.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.327+0000 7f3233fff700 1 -- 172.21.15.43:0/2783673375 >> v1:172.21.15.155:6789/0 conn(0x7f324c101f90 legacy=0x7f324c1973b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:00.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.327+0000 7f3233fff700 1 -- 172.21.15.43:0/2783673375 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f324c1a5af0 con 0x7f324c10a100 2024-03-21T12:52:00.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.327+0000 7f3251748700 1 -- 172.21.15.43:0/2783673375 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f324c1a4b00 con 0x7f324c10a100 2024-03-21T12:52:00.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.327+0000 7f3251748700 1 -- 172.21.15.43:0/2783673375 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f324c1a5040 con 0x7f324c10a100 2024-03-21T12:52:00.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.327+0000 7f3233fff700 1 -- 172.21.15.43:0/2783673375 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f32440013a0 con 0x7f324c10a100 2024-03-21T12:52:00.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.327+0000 7f3233fff700 1 -- 172.21.15.43:0/2783673375 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f3244005180 con 0x7f324c10a100 2024-03-21T12:52:00.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.328+0000 7f3251748700 1 -- 172.21.15.43:0/2783673375 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3238005150 con 0x7f324c10a100 2024-03-21T12:52:00.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.329+0000 7f3233fff700 1 -- 172.21.15.43:0/2783673375 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 16) v1 ==== 92146+0+0 (unknown 510893937 0 0) 0x7f3244005160 con 0x7f324c10a100 2024-03-21T12:52:00.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.329+0000 7f3233fff700 1 -- 172.21.15.43:0/2783673375 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(72..72 src has 1..72) v4 ==== 6001+0+0 (unknown 82262620 0 0) 0x7f3244089ae0 con 0x7f324c10a100 2024-03-21T12:52:00.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.334+0000 7f3233fff700 1 -- 172.21.15.43:0/2783673375 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f3244001e20 con 0x7f324c10a100 2024-03-21T12:52:00.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.627+0000 7f3251748700 1 -- 172.21.15.43:0/2783673375 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f3238005440 con 0x7f324c10a100 2024-03-21T12:52:00.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.629+0000 7f3233fff700 1 -- 172.21.15.43:0/2783673375 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (unknown 57899324 0 1062149991) 0x7f3244057110 con 0x7f324c10a100 2024-03-21T12:52:00.630 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-03-21T12:52:00.630 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQBwLfxlSpWDJRAA1WVBz5oGo9EnWjO06865Hg== 2024-03-21T12:52:00.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.631+0000 7f3251748700 1 -- 172.21.15.43:0/2783673375 >> v1:172.21.15.43:6800/4081968544 conn(0x7f323406f260 legacy=0x7f3234071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:00.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.631+0000 7f3251748700 1 -- 172.21.15.43:0/2783673375 >> v1:172.21.15.43:6789/0 conn(0x7f324c10a100 legacy=0x7f324c1a0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:00.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.631+0000 7f3251748700 1 -- 172.21.15.43:0/2783673375 shutdown_connections 2024-03-21T12:52:00.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.631+0000 7f3251748700 1 -- 172.21.15.43:0/2783673375 >> 172.21.15.43:0/2783673375 conn(0x7f324c0fd530 msgr2=0x7f324c10ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:00.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.631+0000 7f3251748700 1 -- 172.21.15.43:0/2783673375 shutdown_connections 2024-03-21T12:52:00.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:00.631+0000 7f3251748700 1 -- 172.21.15.43:0/2783673375 wait complete. 2024-03-21T12:52:00.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:00 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2783673375' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T12:52:00.937 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:00 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2783673375' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T12:52:00.937 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:00 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2783673375' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T12:52:00.937 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:00 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2783673375' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T12:52:00.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:00 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2783673375' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T12:52:00.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:00 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2783673375' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T12:52:01.453 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-21T12:52:01.453 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-21T12:52:01.453 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-21T12:52:01.499 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-21T12:52:01.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:01 smithi043 ceph-mon[30062]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 41 B/s, 2 objects/s recovering 2024-03-21T12:52:01.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:01 smithi043 ceph-mon[41699]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 41 B/s, 2 objects/s recovering 2024-03-21T12:52:01.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:01 smithi155 ceph-mon[33846]: pgmap v216: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 41 B/s, 2 objects/s recovering 2024-03-21T12:52:02.075 INFO:teuthology.orchestra.run.smithi155.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T12:52:03.865 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:03 smithi155 ceph-mon[33846]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 33 B/s, 1 objects/s recovering 2024-03-21T12:52:03.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:03 smithi043 ceph-mon[30062]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 33 B/s, 1 objects/s recovering 2024-03-21T12:52:03.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:03 smithi043 ceph-mon[41699]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 33 B/s, 1 objects/s recovering 2024-03-21T12:52:05.645 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.644+0000 7f95cd09d700 1 -- 172.21.15.155:0/596669702 >> v1:172.21.15.155:6789/0 conn(0x7f95c8102a70 legacy=0x7f95c8104e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:05.645 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.644+0000 7f95cd09d700 1 -- 172.21.15.155:0/596669702 shutdown_connections 2024-03-21T12:52:05.645 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.644+0000 7f95cd09d700 1 -- 172.21.15.155:0/596669702 >> 172.21.15.155:0/596669702 conn(0x7f95c80fc420 msgr2=0x7f95c80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:05.645 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.644+0000 7f95cd09d700 1 -- 172.21.15.155:0/596669702 shutdown_connections 2024-03-21T12:52:05.645 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.644+0000 7f95cd09d700 1 -- 172.21.15.155:0/596669702 wait complete. 2024-03-21T12:52:05.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.645+0000 7f95cd09d700 1 Processor -- start 2024-03-21T12:52:05.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.645+0000 7f95cd09d700 1 -- start start 2024-03-21T12:52:05.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.645+0000 7f95cd09d700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f95c81981d0 con 0x7f95c8102a70 2024-03-21T12:52:05.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.645+0000 7f95cd09d700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f95c81a39c0 con 0x7f95c8106580 2024-03-21T12:52:05.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.645+0000 7f95cd09d700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f95c81a4ba0 con 0x7f95c810a090 2024-03-21T12:52:05.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.645+0000 7f95bffff700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f95c8106580 0x7f95c819db60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:40100/0 (socket says 172.21.15.155:40100) 2024-03-21T12:52:05.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.645+0000 7f95bffff700 1 -- 172.21.15.155:0/975837810 learned_addr learned my addr 172.21.15.155:0/975837810 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-03-21T12:52:05.647 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.645+0000 7f95c4d99700 1 -- 172.21.15.155:0/975837810 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3737771034 0 0) 0x7f95c81a39c0 con 0x7f95c8106580 2024-03-21T12:52:05.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.645+0000 7f95c4d99700 1 -- 172.21.15.155:0/975837810 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95b40035b0 con 0x7f95c8106580 2024-03-21T12:52:05.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.645+0000 7f95c4d99700 1 -- 172.21.15.155:0/975837810 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1192774578 0 0) 0x7f95b40035b0 con 0x7f95c8106580 2024-03-21T12:52:05.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.646+0000 7f95c4d99700 1 -- 172.21.15.155:0/975837810 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f95c81a39c0 con 0x7f95c8106580 2024-03-21T12:52:05.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.646+0000 7f95c4d99700 1 -- 172.21.15.155:0/975837810 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f95b00018b0 con 0x7f95c8106580 2024-03-21T12:52:05.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.646+0000 7f95c4d99700 1 -- 172.21.15.155:0/975837810 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3760468848 0 0) 0x7f95c81981d0 con 0x7f95c8102a70 2024-03-21T12:52:05.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.646+0000 7f95c4d99700 1 -- 172.21.15.155:0/975837810 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95b40035b0 con 0x7f95c8102a70 2024-03-21T12:52:05.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.646+0000 7f95c4d99700 1 -- 172.21.15.155:0/975837810 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 139974819 0 0) 0x7f95c81a4ba0 con 0x7f95c810a090 2024-03-21T12:52:05.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.646+0000 7f95c4d99700 1 -- 172.21.15.155:0/975837810 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95c81981d0 con 0x7f95c810a090 2024-03-21T12:52:05.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.646+0000 7f95c4d99700 1 -- 172.21.15.155:0/975837810 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1575090272 0 0) 0x7f95c81a39c0 con 0x7f95c8106580 2024-03-21T12:52:05.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.646+0000 7f95c4d99700 1 -- 172.21.15.155:0/975837810 >> v1:172.21.15.43:6790/0 conn(0x7f95c810a090 legacy=0x7f95c81a1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:05.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.646+0000 7f95c4d99700 1 -- 172.21.15.155:0/975837810 >> v1:172.21.15.43:6789/0 conn(0x7f95c8102a70 legacy=0x7f95c8197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:05.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.646+0000 7f95c4d99700 1 -- 172.21.15.155:0/975837810 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c81a5dc0 con 0x7f95c8106580 2024-03-21T12:52:05.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.646+0000 7f95cd09d700 1 -- 172.21.15.155:0/975837810 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95c81a2a10 con 0x7f95c8106580 2024-03-21T12:52:05.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.646+0000 7f95cd09d700 1 -- 172.21.15.155:0/975837810 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f95c81a2f50 con 0x7f95c8106580 2024-03-21T12:52:05.648 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.646+0000 7f95c4d99700 1 -- 172.21.15.155:0/975837810 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f95b0001ce0 con 0x7f95c8106580 2024-03-21T12:52:05.649 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.646+0000 7f95c4d99700 1 -- 172.21.15.155:0/975837810 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f95b0005d80 con 0x7f95c8106580 2024-03-21T12:52:05.649 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.647+0000 7f95cd09d700 1 -- 172.21.15.155:0/975837810 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a8005150 con 0x7f95c8106580 2024-03-21T12:52:05.649 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.647+0000 7f95c4d99700 1 -- 172.21.15.155:0/975837810 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 16) v1 ==== 92146+0+0 (unknown 510893937 0 0) 0x7f95b001c720 con 0x7f95c8106580 2024-03-21T12:52:05.649 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.648+0000 7f95c4d99700 1 -- 172.21.15.155:0/975837810 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(72..72 src has 1..72) v4 ==== 6001+0+0 (unknown 82262620 0 0) 0x7f95b00898f0 con 0x7f95c8106580 2024-03-21T12:52:05.653 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.652+0000 7f95c4d99700 1 -- 172.21.15.155:0/975837810 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f95b0056f20 con 0x7f95c8106580 2024-03-21T12:52:05.809 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:05 smithi155 ceph-mon[33846]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 934 B/s rd, 0 op/s; 30 B/s, 1 objects/s recovering 2024-03-21T12:52:05.929 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.928+0000 7f95cd09d700 1 -- 172.21.15.155:0/975837810 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f95a8005440 con 0x7f95c8106580 2024-03-21T12:52:05.936 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.935+0000 7f95c4d99700 1 -- 172.21.15.155:0/975837810 <== mon.1 v1:172.21.15.155:6789/0 10 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v18) v1 ==== 170+0+59 (unknown 3508017507 0 1164200356) 0x7f95b0056820 con 0x7f95c8106580 2024-03-21T12:52:05.936 INFO:teuthology.orchestra.run.smithi155.stdout:[client.1] 2024-03-21T12:52:05.936 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQB1LfxlcFqCNxAA8kpCPNdXoyrg58uauRaMzg== 2024-03-21T12:52:05.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.938+0000 7f95cd09d700 1 -- 172.21.15.155:0/975837810 >> v1:172.21.15.43:6800/4081968544 conn(0x7f95b406f150 legacy=0x7f95b4071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:05.938 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.938+0000 7f95cd09d700 1 -- 172.21.15.155:0/975837810 >> v1:172.21.15.155:6789/0 conn(0x7f95c8106580 legacy=0x7f95c819db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:05.939 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.938+0000 7f95cd09d700 1 -- 172.21.15.155:0/975837810 shutdown_connections 2024-03-21T12:52:05.939 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.938+0000 7f95cd09d700 1 -- 172.21.15.155:0/975837810 >> 172.21.15.155:0/975837810 conn(0x7f95c80fc420 msgr2=0x7f95c810c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:05.939 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.938+0000 7f95cd09d700 1 -- 172.21.15.155:0/975837810 shutdown_connections 2024-03-21T12:52:05.939 INFO:teuthology.orchestra.run.smithi155.stderr:2024-03-21T12:52:05.938+0000 7f95cd09d700 1 -- 172.21.15.155:0/975837810 wait complete. 2024-03-21T12:52:05.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:05 smithi043 ceph-mon[30062]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 934 B/s rd, 0 op/s; 30 B/s, 1 objects/s recovering 2024-03-21T12:52:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:05 smithi043 ceph-mon[41699]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 934 B/s rd, 0 op/s; 30 B/s, 1 objects/s recovering 2024-03-21T12:52:06.609 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-03-21T12:52:06.609 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-03-21T12:52:06.610 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-21T12:52:06.652 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-21T12:52:06.652 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-21T12:52:06.653 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph mgr dump --format=json 2024-03-21T12:52:06.905 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:52:06.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:06 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.155:0/975837810' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T12:52:06.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:06 smithi155 ceph-mon[33846]: from='client.24493 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T12:52:06.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:06 smithi155 ceph-mon[33846]: from='client.24493 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T12:52:06.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:06 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.155:0/975837810' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T12:52:06.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:06 smithi043 ceph-mon[30062]: from='client.24493 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T12:52:06.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:06 smithi043 ceph-mon[30062]: from='client.24493 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T12:52:06.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:06 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.155:0/975837810' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T12:52:06.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:06 smithi043 ceph-mon[41699]: from='client.24493 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T12:52:06.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:06 smithi043 ceph-mon[41699]: from='client.24493 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T12:52:07.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:07 smithi155 ceph-mon[33846]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 27 B/s, 1 objects/s recovering 2024-03-21T12:52:07.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:07 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:52:07.708 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:52:07 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:52:07.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:07 smithi043 ceph-mon[30062]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 27 B/s, 1 objects/s recovering 2024-03-21T12:52:07.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:07 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:52:07.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:07 smithi043 ceph-mon[41699]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 27 B/s, 1 objects/s recovering 2024-03-21T12:52:07.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:07 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:52:08.148 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:07 smithi155 systemd[1]: Starting Ceph prometheus.a for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T12:52:08.148 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 podman[65289]: 2024-03-21 12:52:08.051789551 +0000 UTC m=+0.019711942 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-03-21T12:52:08.409 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 podman[65289]: 2024-03-21 12:52:08.158092014 +0000 UTC m=+0.126014389 container create ee988b220acfa4567606aaea4308fdf464a7fa6e963c6bcbed4b460b6a676c92 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-21T12:52:08.661 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 podman[65289]: 2024-03-21 12:52:08.410268877 +0000 UTC m=+0.378191256 container init ee988b220acfa4567606aaea4308fdf464a7fa6e963c6bcbed4b460b6a676c92 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-21T12:52:08.661 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 podman[65289]: 2024-03-21 12:52:08.415833926 +0000 UTC m=+0.383756302 container start ee988b220acfa4567606aaea4308fdf464a7fa6e963c6bcbed4b460b6a676c92 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-21T12:52:08.662 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:52:08.440Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-03-21T12:52:08.662 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:52:08.440Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-03-21T12:52:08.662 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:52:08.440Z caller=main.go:570 level=info host_details="(Linux 5.14.0-430.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024 x86_64 smithi155 (none))" 2024-03-21T12:52:08.662 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:52:08.440Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-03-21T12:52:08.662 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:52:08.440Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-03-21T12:52:08.662 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:52:08.441Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-03-21T12:52:08.662 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:52:08.442Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-03-21T12:52:08.662 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:52:08.443Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-03-21T12:52:08.662 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:52:08.443Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-03-21T12:52:08.662 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:52:08.445Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-03-21T12:52:08.662 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:52:08.445Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.693µs 2024-03-21T12:52:08.662 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:52:08.445Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-03-21T12:52:08.662 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:52:08.445Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-03-21T12:52:08.663 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:52:08.445Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=21.834µs wal_replay_duration=246.832µs wbl_replay_duration=107ns total_replay_duration=292.395µs 2024-03-21T12:52:08.663 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:52:08.446Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-03-21T12:52:08.663 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:52:08.446Z caller=main.go:1029 level=info msg="TSDB started" 2024-03-21T12:52:08.663 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:52:08.446Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-03-21T12:52:08.663 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:52:08.467Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=20.96469ms db_storage=882ns remote_storage=1.499µs web_handler=359ns query_engine=513ns scrape=183.192µs scrape_sd=78.248µs notify=735ns notify_sd=1.461µs rules=20.444635ms tracing=6.477µs 2024-03-21T12:52:08.663 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:52:08.467Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-03-21T12:52:08.663 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:52:08.467Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-03-21T12:52:08.663 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 bash[65289]: ee988b220acfa4567606aaea4308fdf464a7fa6e963c6bcbed4b460b6a676c92 2024-03-21T12:52:08.663 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:52:08 smithi155 systemd[1]: Started Ceph prometheus.a for c06c8d1a-e780-11ee-95cd-87774f69a715. 2024-03-21T12:52:08.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.887+0000 7f0058941700 1 -- 172.21.15.43:0/51783178 >> v1:172.21.15.43:6790/0 conn(0x7f0054109d10 legacy=0x7f005410c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:08.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.887+0000 7f0058941700 1 -- 172.21.15.43:0/51783178 shutdown_connections 2024-03-21T12:52:08.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.887+0000 7f0058941700 1 -- 172.21.15.43:0/51783178 >> 172.21.15.43:0/51783178 conn(0x7f00540fc930 msgr2=0x7f00540fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:08.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.887+0000 7f0058941700 1 -- 172.21.15.43:0/51783178 shutdown_connections 2024-03-21T12:52:08.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.887+0000 7f0058941700 1 -- 172.21.15.43:0/51783178 wait complete. 2024-03-21T12:52:08.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.888+0000 7f0058941700 1 Processor -- start 2024-03-21T12:52:08.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.888+0000 7f0058941700 1 -- start start 2024-03-21T12:52:08.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.888+0000 7f0058941700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f00541a6ff0 con 0x7f005410d8b0 2024-03-21T12:52:08.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.888+0000 7f0058941700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f00541a8230 con 0x7f0054109d10 2024-03-21T12:52:08.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.888+0000 7f0058941700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f00541a9470 con 0x7f0054101080 2024-03-21T12:52:08.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.888+0000 7f005259c700 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7f0054101080 0x7f005406a4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:43554/0 (socket says 172.21.15.43:43554) 2024-03-21T12:52:08.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.888+0000 7f005259c700 1 -- 172.21.15.43:0/159136209 learned_addr learned my addr 172.21.15.43:0/159136209 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:52:08.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.888+0000 7f00437fe700 1 -- 172.21.15.43:0/159136209 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3648884372 0 0) 0x7f00541a6ff0 con 0x7f005410d8b0 2024-03-21T12:52:08.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.888+0000 7f00437fe700 1 -- 172.21.15.43:0/159136209 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f003c0031f0 con 0x7f005410d8b0 2024-03-21T12:52:08.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.888+0000 7f00437fe700 1 -- 172.21.15.43:0/159136209 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4036208351 0 0) 0x7f00541a9470 con 0x7f0054101080 2024-03-21T12:52:08.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.888+0000 7f00437fe700 1 -- 172.21.15.43:0/159136209 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00541a6ff0 con 0x7f0054101080 2024-03-21T12:52:08.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.888+0000 7f00437fe700 1 -- 172.21.15.43:0/159136209 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2393791963 0 0) 0x7f003c0031f0 con 0x7f005410d8b0 2024-03-21T12:52:08.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.889+0000 7f00437fe700 1 -- 172.21.15.43:0/159136209 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f00541a9470 con 0x7f005410d8b0 2024-03-21T12:52:08.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.889+0000 7f00437fe700 1 -- 172.21.15.43:0/159136209 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f004c001750 con 0x7f005410d8b0 2024-03-21T12:52:08.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.889+0000 7f00437fe700 1 -- 172.21.15.43:0/159136209 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 391016700 0 0) 0x7f00541a6ff0 con 0x7f0054101080 2024-03-21T12:52:08.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.889+0000 7f00437fe700 1 -- 172.21.15.43:0/159136209 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f003c0031f0 con 0x7f0054101080 2024-03-21T12:52:08.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.889+0000 7f00437fe700 1 -- 172.21.15.43:0/159136209 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568603280 0 0) 0x7f00541a8230 con 0x7f0054109d10 2024-03-21T12:52:08.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.889+0000 7f00437fe700 1 -- 172.21.15.43:0/159136209 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00541a6ff0 con 0x7f0054109d10 2024-03-21T12:52:08.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.889+0000 7f00437fe700 1 -- 172.21.15.43:0/159136209 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f0044001870 con 0x7f0054101080 2024-03-21T12:52:08.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.889+0000 7f00437fe700 1 -- 172.21.15.43:0/159136209 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1863975344 0 0) 0x7f00541a9470 con 0x7f005410d8b0 2024-03-21T12:52:08.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.889+0000 7f00437fe700 1 -- 172.21.15.43:0/159136209 >> v1:172.21.15.43:6790/0 conn(0x7f0054101080 legacy=0x7f005406a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:08.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.889+0000 7f00437fe700 1 -- 172.21.15.43:0/159136209 >> v1:172.21.15.155:6789/0 conn(0x7f0054109d10 legacy=0x7f00541a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:08.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.889+0000 7f00437fe700 1 -- 172.21.15.43:0/159136209 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00541aa6b0 con 0x7f005410d8b0 2024-03-21T12:52:08.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.889+0000 7f0058941700 1 -- 172.21.15.43:0/159136209 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00541a96a0 con 0x7f005410d8b0 2024-03-21T12:52:08.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.889+0000 7f0058941700 1 -- 172.21.15.43:0/159136209 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f00541a9b60 con 0x7f005410d8b0 2024-03-21T12:52:08.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.889+0000 7f00437fe700 1 -- 172.21.15.43:0/159136209 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f004c001a00 con 0x7f005410d8b0 2024-03-21T12:52:08.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.889+0000 7f00437fe700 1 -- 172.21.15.43:0/159136209 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f004c004d50 con 0x7f005410d8b0 2024-03-21T12:52:08.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.890+0000 7f0058941700 1 -- 172.21.15.43:0/159136209 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0034005120 con 0x7f005410d8b0 2024-03-21T12:52:08.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.890+0000 7f00437fe700 1 -- 172.21.15.43:0/159136209 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 16) v1 ==== 92146+0+0 (unknown 510893937 0 0) 0x7f004c003cb0 con 0x7f005410d8b0 2024-03-21T12:52:08.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.891+0000 7f00437fe700 1 -- 172.21.15.43:0/159136209 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(72..72 src has 1..72) v4 ==== 6001+0+0 (unknown 82262620 0 0) 0x7f004c088880 con 0x7f005410d8b0 2024-03-21T12:52:08.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:08.895+0000 7f00437fe700 1 -- 172.21.15.43:0/159136209 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f004c055eb0 con 0x7f005410d8b0 2024-03-21T12:52:08.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:08 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:52:08.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:08 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:52:08.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:08 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:52:08.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:08 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:52:08.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:08 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-21T12:52:08.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:08 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:52:08.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:08 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:52:08.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:08 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:52:08.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:08 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:52:08.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:08 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-21T12:52:08.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:08 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:52:08.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:08 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:52:08.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:08 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:52:08.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:08 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' 2024-03-21T12:52:08.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:08 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-21T12:52:09.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:09.162+0000 7f0058941700 1 -- 172.21.15.43:0/159136209 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f0034005410 con 0x7f005410d8b0 2024-03-21T12:52:09.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:09.165+0000 7f00437fe700 1 -- 172.21.15.43:0/159136209 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v16) v1 ==== 74+0+176176 (unknown 3479712753 0 1698154437) 0x7f004c0557b0 con 0x7f005410d8b0 2024-03-21T12:52:09.168 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:52:09.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:09.167+0000 7f0058941700 1 -- 172.21.15.43:0/159136209 >> v1:172.21.15.43:6800/4081968544 conn(0x7f003c06f0f0 legacy=0x7f003c0715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:09.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:09.167+0000 7f0058941700 1 -- 172.21.15.43:0/159136209 >> v1:172.21.15.43:6789/0 conn(0x7f005410d8b0 legacy=0x7f00541a56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:09.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:09.167+0000 7f0058941700 1 -- 172.21.15.43:0/159136209 shutdown_connections 2024-03-21T12:52:09.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:09.167+0000 7f0058941700 1 -- 172.21.15.43:0/159136209 >> 172.21.15.43:0/159136209 conn(0x7f00540fc930 msgr2=0x7f0054110a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:09.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:09.168+0000 7f0058941700 1 -- 172.21.15.43:0/159136209 shutdown_connections 2024-03-21T12:52:09.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:09.168+0000 7f0058941700 1 -- 172.21.15.43:0/159136209 wait complete. 2024-03-21T12:52:09.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:09 smithi043 ceph-mon[30062]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-03-21T12:52:09.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:09 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/159136209' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-21T12:52:09.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:09 smithi043 ceph-mon[30062]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-21T12:52:09.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:09 smithi043 ceph-mon[30062]: mgrmap e17: y(active, since 5m), standbys: x 2024-03-21T12:52:09.734 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:09 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ignoring --setuser ceph since I am not root 2024-03-21T12:52:09.734 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:09 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ignoring --setgroup ceph since I am not root 2024-03-21T12:52:09.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:09 smithi043 ceph-mon[41699]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-03-21T12:52:09.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:09 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/159136209' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-21T12:52:09.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:09 smithi043 ceph-mon[41699]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-21T12:52:09.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:09 smithi043 ceph-mon[41699]: mgrmap e17: y(active, since 5m), standbys: x 2024-03-21T12:52:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:09 smithi155 ceph-mon[33846]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 1 objects/s recovering 2024-03-21T12:52:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:09 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/159136209' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-21T12:52:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:09 smithi155 ceph-mon[33846]: from='mgr.14152 v1:172.21.15.43:0/3176867570' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-21T12:52:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:09 smithi155 ceph-mon[33846]: mgrmap e17: y(active, since 5m), standbys: x 2024-03-21T12:52:09.957 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:09 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: ignoring --setuser ceph since I am not root 2024-03-21T12:52:09.958 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:09 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: ignoring --setgroup ceph since I am not root 2024-03-21T12:52:10.031 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":16,"flags":0,"active_gid":14152,"active_name":"y","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6800","nonce":4081968544}]},"active_addr":"172.21.15.43:6800/4081968544","active_change":"2024-03-21T12:46:51.885554+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.43:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":2437133700}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":2176767283}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":902871424}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":2775613288}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.43:0","nonce":485500436}]}]} 2024-03-21T12:52:10.035 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-21T12:52:10.035 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-21T12:52:10.035 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-21T12:52:10.306 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:52:10.310 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:10 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:10.192+0000 7f8bc327d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T12:52:10.311 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:10 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:10.309+0000 7f8bc327d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T12:52:10.457 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:10 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:10.163+0000 7f86e012c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T12:52:10.457 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:10 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:10.256+0000 7f86e012c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T12:52:11.117 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:11 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:11.002+0000 7f8bc327d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T12:52:11.117 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:11 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:11.098+0000 7f8bc327d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T12:52:11.207 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:10 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:10.896+0000 7f86e012c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T12:52:11.207 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:10 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:10.990+0000 7f86e012c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T12:52:11.207 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:11 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:11.133+0000 7f86e012c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T12:52:11.426 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:11 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:11.244+0000 7f8bc327d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T12:52:11.426 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:11 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:11.341+0000 7f8bc327d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T12:52:11.582 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:11 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:11.230+0000 7f86e012c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T12:52:11.582 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:11 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:11.314+0000 7f86e012c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T12:52:11.726 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:11 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:11.428+0000 7f8bc327d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T12:52:11.873 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:11 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:11.581+0000 7f86e012c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T12:52:11.873 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:11 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:11.692+0000 7f86e012c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T12:52:11.873 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:11 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:11.777+0000 7f86e012c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T12:52:11.873 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:11 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:11.872+0000 7f86e012c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T12:52:11.984 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:11 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:11.725+0000 7f8bc327d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T12:52:11.984 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:11 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:11.838+0000 7f8bc327d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T12:52:11.984 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:11 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:11.931+0000 7f8bc327d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T12:52:12.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.122+0000 7f3470bd7700 1 -- 172.21.15.43:0/3849739374 >> v1:172.21.15.43:6790/0 conn(0x7f346c101080 legacy=0x7f346c101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:12.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.122+0000 7f3470bd7700 1 -- 172.21.15.43:0/3849739374 shutdown_connections 2024-03-21T12:52:12.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.122+0000 7f3470bd7700 1 -- 172.21.15.43:0/3849739374 >> 172.21.15.43:0/3849739374 conn(0x7f346c0fc930 msgr2=0x7f346c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:12.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.122+0000 7f3470bd7700 1 -- 172.21.15.43:0/3849739374 shutdown_connections 2024-03-21T12:52:12.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.122+0000 7f3470bd7700 1 -- 172.21.15.43:0/3849739374 wait complete. 2024-03-21T12:52:12.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.123+0000 7f3470bd7700 1 Processor -- start 2024-03-21T12:52:12.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.123+0000 7f3470bd7700 1 -- start start 2024-03-21T12:52:12.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.123+0000 7f3470bd7700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f346c1a7000 con 0x7f346c10d8b0 2024-03-21T12:52:12.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.123+0000 7f3470bd7700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f346c1a8220 con 0x7f346c101080 2024-03-21T12:52:12.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.123+0000 7f3470bd7700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f346c1a9460 con 0x7f346c109d10 2024-03-21T12:52:12.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.123+0000 7f346ad9d700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f346c10d8b0 0x7f346c1a56c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55944/0 (socket says 172.21.15.43:55944) 2024-03-21T12:52:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.123+0000 7f346ad9d700 1 -- 172.21.15.43:0/2539172013 learned_addr learned my addr 172.21.15.43:0/2539172013 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:52:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.124+0000 7f34637fe700 1 -- 172.21.15.43:0/2539172013 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1943456071 0 0) 0x7f346c1a8220 con 0x7f346c101080 2024-03-21T12:52:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.124+0000 7f34637fe700 1 -- 172.21.15.43:0/2539172013 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34580031f0 con 0x7f346c101080 2024-03-21T12:52:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.124+0000 7f34637fe700 1 -- 172.21.15.43:0/2539172013 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1564623322 0 0) 0x7f346c1a9460 con 0x7f346c109d10 2024-03-21T12:52:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.124+0000 7f34637fe700 1 -- 172.21.15.43:0/2539172013 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f346c1a8220 con 0x7f346c109d10 2024-03-21T12:52:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.124+0000 7f34637fe700 1 -- 172.21.15.43:0/2539172013 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3137422860 0 0) 0x7f346c1a7000 con 0x7f346c10d8b0 2024-03-21T12:52:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.124+0000 7f34637fe700 1 -- 172.21.15.43:0/2539172013 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f346c1a9460 con 0x7f346c10d8b0 2024-03-21T12:52:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.124+0000 7f34637fe700 1 -- 172.21.15.43:0/2539172013 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4088073511 0 0) 0x7f34580031f0 con 0x7f346c101080 2024-03-21T12:52:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.124+0000 7f34637fe700 1 -- 172.21.15.43:0/2539172013 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f346c1a7000 con 0x7f346c101080 2024-03-21T12:52:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.124+0000 7f34637fe700 1 -- 172.21.15.43:0/2539172013 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f345c001a00 con 0x7f346c101080 2024-03-21T12:52:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.124+0000 7f34637fe700 1 -- 172.21.15.43:0/2539172013 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4003009587 0 0) 0x7f346c1a9460 con 0x7f346c10d8b0 2024-03-21T12:52:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.124+0000 7f34637fe700 1 -- 172.21.15.43:0/2539172013 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f34580031f0 con 0x7f346c10d8b0 2024-03-21T12:52:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.124+0000 7f34637fe700 1 -- 172.21.15.43:0/2539172013 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 308322248 0 0) 0x7f346c1a8220 con 0x7f346c109d10 2024-03-21T12:52:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.124+0000 7f34637fe700 1 -- 172.21.15.43:0/2539172013 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f346c1a9460 con 0x7f346c109d10 2024-03-21T12:52:12.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.124+0000 7f34637fe700 1 -- 172.21.15.43:0/2539172013 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f3464001b40 con 0x7f346c10d8b0 2024-03-21T12:52:12.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.124+0000 7f34637fe700 1 -- 172.21.15.43:0/2539172013 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f3454001870 con 0x7f346c109d10 2024-03-21T12:52:12.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.124+0000 7f34637fe700 1 -- 172.21.15.43:0/2539172013 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3669522868 0 0) 0x7f346c1a7000 con 0x7f346c101080 2024-03-21T12:52:12.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.124+0000 7f34637fe700 1 -- 172.21.15.43:0/2539172013 >> v1:172.21.15.43:6790/0 conn(0x7f346c109d10 legacy=0x7f346c1a1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:12.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.124+0000 7f34637fe700 1 -- 172.21.15.43:0/2539172013 >> v1:172.21.15.43:6789/0 conn(0x7f346c10d8b0 legacy=0x7f346c1a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:12.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.124+0000 7f34637fe700 1 -- 172.21.15.43:0/2539172013 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f346c1aa6a0 con 0x7f346c101080 2024-03-21T12:52:12.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.124+0000 7f3470bd7700 1 -- 172.21.15.43:0/2539172013 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f346c1a8450 con 0x7f346c101080 2024-03-21T12:52:12.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.124+0000 7f3470bd7700 1 -- 172.21.15.43:0/2539172013 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f346c1a8910 con 0x7f346c101080 2024-03-21T12:52:12.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.124+0000 7f34637fe700 1 -- 172.21.15.43:0/2539172013 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f345c001cf0 con 0x7f346c101080 2024-03-21T12:52:12.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.124+0000 7f34637fe700 1 -- 172.21.15.43:0/2539172013 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f345c004c30 con 0x7f346c101080 2024-03-21T12:52:12.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.125+0000 7f3470bd7700 1 -- 172.21.15.43:0/2539172013 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f344c005120 con 0x7f346c101080 2024-03-21T12:52:12.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.126+0000 7f34637fe700 1 -- 172.21.15.43:0/2539172013 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 17) v1 ==== 92160+0+0 (unknown 1287971880 0 0) 0x7f345c01b660 con 0x7f346c101080 2024-03-21T12:52:12.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.126+0000 7f3469d9b700 1 -- 172.21.15.43:0/2539172013 >> v1:172.21.15.43:6800/4081968544 conn(0x7f345806f170 legacy=0x7f3458071630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.43:6800/4081968544 2024-03-21T12:52:12.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.126+0000 7f34637fe700 1 -- 172.21.15.43:0/2539172013 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(72..72 src has 1..72) v4 ==== 6001+0+0 (unknown 82262620 0 0) 0x7f345c0885b0 con 0x7f346c101080 2024-03-21T12:52:12.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.131+0000 7f34637fe700 1 -- 172.21.15.43:0/2539172013 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f345c055b20 con 0x7f346c101080 2024-03-21T12:52:12.207 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:12 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:12.047+0000 7f86e012c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T12:52:12.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.327+0000 7f3469d9b700 1 -- 172.21.15.43:0/2539172013 >> v1:172.21.15.43:6800/4081968544 conn(0x7f345806f170 legacy=0x7f3458071630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.43:6800/4081968544 2024-03-21T12:52:12.328 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:12 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:12.029+0000 7f8bc327d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T12:52:12.328 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:12 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:12.213+0000 7f8bc327d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T12:52:12.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.338+0000 7f3470bd7700 1 -- 172.21.15.43:0/2539172013 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f344c005410 con 0x7f346c101080 2024-03-21T12:52:12.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.340+0000 7f34637fe700 1 -- 172.21.15.43:0/2539172013 <== mon.1 v1:172.21.15.155:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+20679 (unknown 1686360691 0 555127514) 0x7f345c055420 con 0x7f346c101080 2024-03-21T12:52:12.341 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:52:12.341 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":72,"fsid":"c06c8d1a-e780-11ee-95cd-87774f69a715","created":"2024-03-21T12:45:46.690508+0000","modified":"2024-03-21T12:51:52.978766+0000","last_up_change":"2024-03-21T12:51:21.077174+0000","last_in_change":"2024-03-21T12:51:02.931933+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T12:49:12.226146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-03-21T12:51:33.998346+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":65,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-03-21T12:51:34.456369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-03-21T12:51:36.382682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-03-21T12:51:38.345178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-03-21T12:51:40.399455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c030e54-4f13-460e-8ec3-940135cee693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6801","nonce":3260944688}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6802","nonce":3260944688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6804","nonce":3260944688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6803","nonce":3260944688}]},"public_addr":"172.21.15.43:6801/3260944688","cluster_addr":"172.21.15.43:6802/3260944688","heartbeat_back_addr":"172.21.15.43:6804/3260944688","heartbeat_front_addr":"172.21.15.43:6803/3260944688","state":["exists","up"]},{"osd":1,"uuid":"fcc9c3e0-665d-4331-abf3-3ea6e2ac2c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6805","nonce":1862805780}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6806","nonce":1862805780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6808","nonce":1862805780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6807","nonce":1862805780}]},"public_addr":"172.21.15.43:6805/1862805780","cluster_addr":"172.21.15.43:6806/1862805780","heartbeat_back_addr":"172.21.15.43:6808/1862805780","heartbeat_front_addr":"172.21.15.43:6807/1862805780","state":["exists","up"]},{"osd":2,"uuid":"63230fe0-2225-4ebd-a6de-aa55fc7ea0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6809","nonce":1358463812}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6810","nonce":1358463812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6812","nonce":1358463812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6811","nonce":1358463812}]},"public_addr":"172.21.15.43:6809/1358463812","cluster_addr":"172.21.15.43:6810/1358463812","heartbeat_back_addr":"172.21.15.43:6812/1358463812","heartbeat_front_addr":"172.21.15.43:6811/1358463812","state":["exists","up"]},{"osd":3,"uuid":"ac04f1e6-c31c-44b0-909d-a8679d39b335","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6813","nonce":2246303296}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6814","nonce":2246303296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6816","nonce":2246303296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6815","nonce":2246303296}]},"public_addr":"172.21.15.43:6813/2246303296","cluster_addr":"172.21.15.43:6814/2246303296","heartbeat_back_addr":"172.21.15.43:6816/2246303296","heartbeat_front_addr":"172.21.15.43:6815/2246303296","state":["exists","up"]},{"osd":4,"uuid":"28d67123-5c3e-4973-b1dc-848eb829c590","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6800","nonce":2961807269}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6801","nonce":2961807269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6803","nonce":2961807269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6802","nonce":2961807269}]},"public_addr":"172.21.15.155:6800/2961807269","cluster_addr":"172.21.15.155:6801/2961807269","heartbeat_back_addr":"172.21.15.155:6803/2961807269","heartbeat_front_addr":"172.21.15.155:6802/2961807269","state":["exists","up"]},{"osd":5,"uuid":"3643dae0-6e18-4261-9d47-b3f7606e59e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6804","nonce":1838740519}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6805","nonce":1838740519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6807","nonce":1838740519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6806","nonce":1838740519}]},"public_addr":"172.21.15.155:6804/1838740519","cluster_addr":"172.21.15.155:6805/1838740519","heartbeat_back_addr":"172.21.15.155:6807/1838740519","heartbeat_front_addr":"172.21.15.155:6806/1838740519","state":["exists","up"]},{"osd":6,"uuid":"45f89c89-8fa1-455d-9278-8c7ddf0bc422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6808","nonce":2978803316}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6809","nonce":2978803316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6811","nonce":2978803316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6810","nonce":2978803316}]},"public_addr":"172.21.15.155:6808/2978803316","cluster_addr":"172.21.15.155:6809/2978803316","heartbeat_back_addr":"172.21.15.155:6811/2978803316","heartbeat_front_addr":"172.21.15.155:6810/2978803316","state":["exists","up"]},{"osd":7,"uuid":"56960e89-0f80-4192-95cb-54cf6da32ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6812","nonce":226070080}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6813","nonce":226070080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6815","nonce":226070080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6814","nonce":226070080}]},"public_addr":"172.21.15.155:6812/226070080","cluster_addr":"172.21.15.155:6813/226070080","heartbeat_back_addr":"172.21.15.155:6815/226070080","heartbeat_front_addr":"172.21.15.155:6814/226070080","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T12:48:17.270391+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T12:48:43.552605+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T12:49:09.886944+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T12:49:36.259542+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T12:50:00.234269+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T12:50:26.271712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T12:50:53.074703+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T12:51:19.360649+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/2708116611":"2024-03-22T12:46:51.885437+0000","172.21.15.43:6800/2056453291":"2024-03-22T12:46:23.703537+0000","172.21.15.43:6800/1388905519":"2024-03-22T12:46:51.885437+0000","172.21.15.43:0/1502364308":"2024-03-22T12:46:23.703537+0000","172.21.15.43:0/404209980":"2024-03-22T12:46:51.885437+0000","172.21.15.43:0/1693637100":"2024-03-22T12:46:23.703537+0000","172.21.15.43:0/2906079211":"2024-03-22T12:46:23.703537+0000","172.21.15.43:0/1705359135":"2024-03-22T12:46:51.885437+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T12:52:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.342+0000 7f3470bd7700 1 -- 172.21.15.43:0/2539172013 >> v1:172.21.15.43:6800/4081968544 conn(0x7f345806f170 legacy=0x7f3458071630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T12:52:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.342+0000 7f3470bd7700 1 -- 172.21.15.43:0/2539172013 >> v1:172.21.15.155:6789/0 conn(0x7f346c101080 legacy=0x7f346c1063c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.342+0000 7f3470bd7700 1 -- 172.21.15.43:0/2539172013 shutdown_connections 2024-03-21T12:52:12.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.342+0000 7f3470bd7700 1 -- 172.21.15.43:0/2539172013 >> 172.21.15.43:0/2539172013 conn(0x7f346c0fc930 msgr2=0x7f346c06a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:12.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.342+0000 7f3470bd7700 1 -- 172.21.15.43:0/2539172013 shutdown_connections 2024-03-21T12:52:12.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:12.342+0000 7f3470bd7700 1 -- 172.21.15.43:0/2539172013 wait complete. 2024-03-21T12:52:12.650 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:12 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2539172013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T12:52:12.651 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:12 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2539172013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T12:52:12.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:12 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2539172013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T12:52:12.929 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-21T12:52:12.930 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-21T12:52:13.177 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:52:13.457 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:13 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:13.009+0000 7f86e012c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T12:52:13.457 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:13 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:13.140+0000 7f86e012c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T12:52:13.596 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:13 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:13.233+0000 7f8bc327d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T12:52:13.596 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:13 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:13.369+0000 7f8bc327d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T12:52:13.754 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:13 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:13.471+0000 7f86e012c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T12:52:13.754 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:13 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:13.560+0000 7f86e012c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T12:52:13.754 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:13 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:13.754+0000 7f86e012c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T12:52:13.983 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:13 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:13.717+0000 7f8bc327d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T12:52:13.984 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:13 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:13.810+0000 7f8bc327d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T12:52:14.207 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:13 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:13.939+0000 7f86e012c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T12:52:14.326 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:14 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:14.013+0000 7f8bc327d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T12:52:14.327 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:14 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:14.208+0000 7f8bc327d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T12:52:14.485 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:14 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:14.223+0000 7f86e012c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T12:52:14.640 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:14 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:14.505+0000 7f8bc327d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T12:52:14.757 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:14 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:14.485+0000 7f86e012c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T12:52:14.757 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:14 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:14.724+0000 7f86e012c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T12:52:14.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.895+0000 7fb644d82700 1 -- 172.21.15.43:0/4100633571 >> v1:172.21.15.43:6789/0 conn(0x7fb6401021a0 legacy=0x7fb640102580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:14.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.895+0000 7fb644d82700 1 -- 172.21.15.43:0/4100633571 shutdown_connections 2024-03-21T12:52:14.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.895+0000 7fb644d82700 1 -- 172.21.15.43:0/4100633571 >> 172.21.15.43:0/4100633571 conn(0x7fb6400fda30 msgr2=0x7fb6400ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:14.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.895+0000 7fb644d82700 1 -- 172.21.15.43:0/4100633571 shutdown_connections 2024-03-21T12:52:14.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.895+0000 7fb644d82700 1 -- 172.21.15.43:0/4100633571 wait complete. 2024-03-21T12:52:14.897 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:14 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:14.771+0000 7f8bc327d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T12:52:14.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.896+0000 7fb644d82700 1 Processor -- start 2024-03-21T12:52:14.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.896+0000 7fb644d82700 1 -- start start 2024-03-21T12:52:14.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.896+0000 7fb644d82700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb6401a91e0 con 0x7fb64010e9d0 2024-03-21T12:52:14.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.896+0000 7fb644d82700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb6401aa400 con 0x7fb6401021a0 2024-03-21T12:52:14.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.896+0000 7fb644d82700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb6401ab640 con 0x7fb64010ae30 2024-03-21T12:52:14.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.896+0000 7fb63dd9b700 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7fb64010ae30 0x7fb6401a4230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:58436/0 (socket says 172.21.15.43:58436) 2024-03-21T12:52:14.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.896+0000 7fb63dd9b700 1 -- 172.21.15.43:0/2283143282 learned_addr learned my addr 172.21.15.43:0/2283143282 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:52:14.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.896+0000 7fb62f7fe700 1 -- 172.21.15.43:0/2283143282 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378359611 0 0) 0x7fb6401a91e0 con 0x7fb64010e9d0 2024-03-21T12:52:14.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.896+0000 7fb62f7fe700 1 -- 172.21.15.43:0/2283143282 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6280035b0 con 0x7fb64010e9d0 2024-03-21T12:52:14.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.896+0000 7fb62f7fe700 1 -- 172.21.15.43:0/2283143282 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 669125129 0 0) 0x7fb6401ab640 con 0x7fb64010ae30 2024-03-21T12:52:14.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.896+0000 7fb62f7fe700 1 -- 172.21.15.43:0/2283143282 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6401a91e0 con 0x7fb64010ae30 2024-03-21T12:52:14.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.896+0000 7fb62f7fe700 1 -- 172.21.15.43:0/2283143282 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3518010721 0 0) 0x7fb6401aa400 con 0x7fb6401021a0 2024-03-21T12:52:14.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.896+0000 7fb62f7fe700 1 -- 172.21.15.43:0/2283143282 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6401ab640 con 0x7fb6401021a0 2024-03-21T12:52:14.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.897+0000 7fb62f7fe700 1 -- 172.21.15.43:0/2283143282 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3448323617 0 0) 0x7fb6401ab640 con 0x7fb6401021a0 2024-03-21T12:52:14.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.897+0000 7fb62f7fe700 1 -- 172.21.15.43:0/2283143282 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb6401aa400 con 0x7fb6401021a0 2024-03-21T12:52:14.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.897+0000 7fb62f7fe700 1 -- 172.21.15.43:0/2283143282 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fb630004420 con 0x7fb6401021a0 2024-03-21T12:52:14.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.897+0000 7fb62f7fe700 1 -- 172.21.15.43:0/2283143282 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3294166458 0 0) 0x7fb6280035b0 con 0x7fb64010e9d0 2024-03-21T12:52:14.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.897+0000 7fb62f7fe700 1 -- 172.21.15.43:0/2283143282 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb628002640 con 0x7fb64010e9d0 2024-03-21T12:52:14.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.897+0000 7fb62f7fe700 1 -- 172.21.15.43:0/2283143282 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2381761303 0 0) 0x7fb6401a91e0 con 0x7fb64010ae30 2024-03-21T12:52:14.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.897+0000 7fb62f7fe700 1 -- 172.21.15.43:0/2283143282 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb6280035b0 con 0x7fb64010ae30 2024-03-21T12:52:14.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.897+0000 7fb62f7fe700 1 -- 172.21.15.43:0/2283143282 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fb638001b20 con 0x7fb64010e9d0 2024-03-21T12:52:14.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.897+0000 7fb62f7fe700 1 -- 172.21.15.43:0/2283143282 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fb634001870 con 0x7fb64010ae30 2024-03-21T12:52:14.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.897+0000 7fb62f7fe700 1 -- 172.21.15.43:0/2283143282 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 81836561 0 0) 0x7fb6401aa400 con 0x7fb6401021a0 2024-03-21T12:52:14.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.897+0000 7fb62f7fe700 1 -- 172.21.15.43:0/2283143282 >> v1:172.21.15.43:6790/0 conn(0x7fb64010ae30 legacy=0x7fb6401a4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:14.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.897+0000 7fb62f7fe700 1 -- 172.21.15.43:0/2283143282 >> v1:172.21.15.43:6789/0 conn(0x7fb64010e9d0 legacy=0x7fb6401a7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:14.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.897+0000 7fb62f7fe700 1 -- 172.21.15.43:0/2283143282 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6401ac880 con 0x7fb6401021a0 2024-03-21T12:52:14.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.897+0000 7fb644d82700 1 -- 172.21.15.43:0/2283143282 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6401ab870 con 0x7fb6401021a0 2024-03-21T12:52:14.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.897+0000 7fb644d82700 1 -- 172.21.15.43:0/2283143282 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6401abd30 con 0x7fb6401021a0 2024-03-21T12:52:14.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.897+0000 7fb62f7fe700 1 -- 172.21.15.43:0/2283143282 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7fb630001b70 con 0x7fb6401021a0 2024-03-21T12:52:14.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.897+0000 7fb62f7fe700 1 -- 172.21.15.43:0/2283143282 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fb630004bb0 con 0x7fb6401021a0 2024-03-21T12:52:14.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.898+0000 7fb644d82700 1 -- 172.21.15.43:0/2283143282 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb620005120 con 0x7fb6401021a0 2024-03-21T12:52:14.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.899+0000 7fb62f7fe700 1 -- 172.21.15.43:0/2283143282 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 17) v1 ==== 92160+0+0 (unknown 1287971880 0 0) 0x7fb63001b560 con 0x7fb6401021a0 2024-03-21T12:52:14.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.899+0000 7fb63dd9b700 1 -- 172.21.15.43:0/2283143282 >> v1:172.21.15.43:6800/4081968544 conn(0x7fb62806f450 legacy=0x7fb628071910 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.43:6800/4081968544 2024-03-21T12:52:14.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.899+0000 7fb62f7fe700 1 -- 172.21.15.43:0/2283143282 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(72..72 src has 1..72) v4 ==== 6001+0+0 (unknown 82262620 0 0) 0x7fb630088660 con 0x7fb6401021a0 2024-03-21T12:52:14.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:14.904+0000 7fb62f7fe700 1 -- 172.21.15.43:0/2283143282 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7fb630055bd0 con 0x7fb6401021a0 2024-03-21T12:52:15.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:15.100+0000 7fb63dd9b700 1 -- 172.21.15.43:0/2283143282 >> v1:172.21.15.43:6800/4081968544 conn(0x7fb62806f450 legacy=0x7fb628071910 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.43:6800/4081968544 2024-03-21T12:52:15.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:15.113+0000 7fb644d82700 1 -- 172.21.15.43:0/2283143282 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb620005410 con 0x7fb6401021a0 2024-03-21T12:52:15.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:15.114+0000 7fb62f7fe700 1 -- 172.21.15.43:0/2283143282 <== mon.1 v1:172.21.15.155:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+20679 (unknown 1686360691 0 555127514) 0x7fb6300554d0 con 0x7fb6401021a0 2024-03-21T12:52:15.116 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:52:15.116 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":72,"fsid":"c06c8d1a-e780-11ee-95cd-87774f69a715","created":"2024-03-21T12:45:46.690508+0000","modified":"2024-03-21T12:51:52.978766+0000","last_up_change":"2024-03-21T12:51:21.077174+0000","last_in_change":"2024-03-21T12:51:02.931933+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T12:49:12.226146+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-03-21T12:51:33.998346+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":65,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-03-21T12:51:34.456369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-03-21T12:51:36.382682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-03-21T12:51:38.345178+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-03-21T12:51:40.399455+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3c030e54-4f13-460e-8ec3-940135cee693","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6801","nonce":3260944688}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6802","nonce":3260944688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6804","nonce":3260944688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6803","nonce":3260944688}]},"public_addr":"172.21.15.43:6801/3260944688","cluster_addr":"172.21.15.43:6802/3260944688","heartbeat_back_addr":"172.21.15.43:6804/3260944688","heartbeat_front_addr":"172.21.15.43:6803/3260944688","state":["exists","up"]},{"osd":1,"uuid":"fcc9c3e0-665d-4331-abf3-3ea6e2ac2c63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6805","nonce":1862805780}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6806","nonce":1862805780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6808","nonce":1862805780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6807","nonce":1862805780}]},"public_addr":"172.21.15.43:6805/1862805780","cluster_addr":"172.21.15.43:6806/1862805780","heartbeat_back_addr":"172.21.15.43:6808/1862805780","heartbeat_front_addr":"172.21.15.43:6807/1862805780","state":["exists","up"]},{"osd":2,"uuid":"63230fe0-2225-4ebd-a6de-aa55fc7ea0b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6809","nonce":1358463812}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6810","nonce":1358463812}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6812","nonce":1358463812}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6811","nonce":1358463812}]},"public_addr":"172.21.15.43:6809/1358463812","cluster_addr":"172.21.15.43:6810/1358463812","heartbeat_back_addr":"172.21.15.43:6812/1358463812","heartbeat_front_addr":"172.21.15.43:6811/1358463812","state":["exists","up"]},{"osd":3,"uuid":"ac04f1e6-c31c-44b0-909d-a8679d39b335","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6813","nonce":2246303296}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6814","nonce":2246303296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6816","nonce":2246303296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.43:6815","nonce":2246303296}]},"public_addr":"172.21.15.43:6813/2246303296","cluster_addr":"172.21.15.43:6814/2246303296","heartbeat_back_addr":"172.21.15.43:6816/2246303296","heartbeat_front_addr":"172.21.15.43:6815/2246303296","state":["exists","up"]},{"osd":4,"uuid":"28d67123-5c3e-4973-b1dc-848eb829c590","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6800","nonce":2961807269}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6801","nonce":2961807269}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6803","nonce":2961807269}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6802","nonce":2961807269}]},"public_addr":"172.21.15.155:6800/2961807269","cluster_addr":"172.21.15.155:6801/2961807269","heartbeat_back_addr":"172.21.15.155:6803/2961807269","heartbeat_front_addr":"172.21.15.155:6802/2961807269","state":["exists","up"]},{"osd":5,"uuid":"3643dae0-6e18-4261-9d47-b3f7606e59e1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6804","nonce":1838740519}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6805","nonce":1838740519}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6807","nonce":1838740519}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6806","nonce":1838740519}]},"public_addr":"172.21.15.155:6804/1838740519","cluster_addr":"172.21.15.155:6805/1838740519","heartbeat_back_addr":"172.21.15.155:6807/1838740519","heartbeat_front_addr":"172.21.15.155:6806/1838740519","state":["exists","up"]},{"osd":6,"uuid":"45f89c89-8fa1-455d-9278-8c7ddf0bc422","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6808","nonce":2978803316}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6809","nonce":2978803316}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6811","nonce":2978803316}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6810","nonce":2978803316}]},"public_addr":"172.21.15.155:6808/2978803316","cluster_addr":"172.21.15.155:6809/2978803316","heartbeat_back_addr":"172.21.15.155:6811/2978803316","heartbeat_front_addr":"172.21.15.155:6810/2978803316","state":["exists","up"]},{"osd":7,"uuid":"56960e89-0f80-4192-95cb-54cf6da32ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6812","nonce":226070080}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6813","nonce":226070080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6815","nonce":226070080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6814","nonce":226070080}]},"public_addr":"172.21.15.155:6812/226070080","cluster_addr":"172.21.15.155:6813/226070080","heartbeat_back_addr":"172.21.15.155:6815/226070080","heartbeat_front_addr":"172.21.15.155:6814/226070080","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T12:48:17.270391+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T12:48:43.552605+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T12:49:09.886944+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T12:49:36.259542+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T12:50:00.234269+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T12:50:26.271712+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T12:50:53.074703+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T12:51:19.360649+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.9","mappings":[{"from":1,"to":2}]},{"pgid":"4.1c","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:0/2708116611":"2024-03-22T12:46:51.885437+0000","172.21.15.43:6800/2056453291":"2024-03-22T12:46:23.703537+0000","172.21.15.43:6800/1388905519":"2024-03-22T12:46:51.885437+0000","172.21.15.43:0/1502364308":"2024-03-22T12:46:23.703537+0000","172.21.15.43:0/404209980":"2024-03-22T12:46:51.885437+0000","172.21.15.43:0/1693637100":"2024-03-22T12:46:23.703537+0000","172.21.15.43:0/2906079211":"2024-03-22T12:46:23.703537+0000","172.21.15.43:0/1705359135":"2024-03-22T12:46:51.885437+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T12:52:15.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:15.117+0000 7fb644d82700 1 -- 172.21.15.43:0/2283143282 >> v1:172.21.15.43:6800/4081968544 conn(0x7fb62806f450 legacy=0x7fb628071910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T12:52:15.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:15.117+0000 7fb644d82700 1 -- 172.21.15.43:0/2283143282 >> v1:172.21.15.155:6789/0 conn(0x7fb6401021a0 legacy=0x7fb640107410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:15.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:15.117+0000 7fb644d82700 1 -- 172.21.15.43:0/2283143282 shutdown_connections 2024-03-21T12:52:15.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:15.117+0000 7fb644d82700 1 -- 172.21.15.43:0/2283143282 >> 172.21.15.43:0/2283143282 conn(0x7fb6400fda30 msgr2=0x7fb6401116e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:15.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:15.117+0000 7fb644d82700 1 -- 172.21.15.43:0/2283143282 shutdown_connections 2024-03-21T12:52:15.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:15.117+0000 7fb644d82700 1 -- 172.21.15.43:0/2283143282 wait complete. 2024-03-21T12:52:15.167 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:15 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:15.025+0000 7f8bc327d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T12:52:15.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:15 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2283143282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T12:52:15.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:15 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2283143282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T12:52:15.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:15 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2283143282' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T12:52:15.819 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-21T12:52:15.820 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-21T12:52:15.820 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-21T12:52:15.821 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-21T12:52:15.821 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-21T12:52:15.821 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-21T12:52:15.822 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-21T12:52:15.822 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-21T12:52:15.957 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:15.548+0000 7f86e012c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T12:52:16.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:15 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:15.907+0000 7f8bc327d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T12:52:16.363 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:52:16.364 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:52:16.371 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:52:16.371 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:52:16.371 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:52:16.374 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:52:16.380 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:52:16.440 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:16.123+0000 7f86e012c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T12:52:16.441 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:16.222+0000 7f86e012c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T12:52:16.441 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:16.440+0000 7f86e012c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T12:52:16.678 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:52:16.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:16 smithi155 ceph-mon[33846]: Standby manager daemon x restarted 2024-03-21T12:52:16.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:16 smithi155 ceph-mon[33846]: Standby manager daemon x started 2024-03-21T12:52:16.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:16 smithi155 ceph-mon[33846]: from='mgr.? v1:172.21.15.155:0/1541130107' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-21T12:52:16.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:16 smithi155 ceph-mon[33846]: from='mgr.? v1:172.21.15.155:0/1541130107' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T12:52:16.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:16 smithi155 ceph-mon[33846]: from='mgr.? v1:172.21.15.155:0/1541130107' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-21T12:52:16.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:16 smithi155 ceph-mon[33846]: from='mgr.? v1:172.21.15.155:0/1541130107' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T12:52:16.708 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: [21/Mar/2024:12:52:16] ENGINE Bus STARTING 2024-03-21T12:52:16.708 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: CherryPy Checker: 2024-03-21T12:52:16.708 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: The Application mounted at '' has an empty config. 2024-03-21T12:52:16.708 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: 2024-03-21T12:52:16.708 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: [21/Mar/2024:12:52:16] ENGINE Serving on http://:::9283 2024-03-21T12:52:16.708 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 12:52:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x[35923]: [21/Mar/2024:12:52:16] ENGINE Bus STARTED 2024-03-21T12:52:16.770 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:16 smithi043 ceph-mon[30062]: Standby manager daemon x restarted 2024-03-21T12:52:16.771 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:16 smithi043 ceph-mon[30062]: Standby manager daemon x started 2024-03-21T12:52:16.771 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:16 smithi043 ceph-mon[30062]: from='mgr.? v1:172.21.15.155:0/1541130107' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-21T12:52:16.771 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:16 smithi043 ceph-mon[30062]: from='mgr.? v1:172.21.15.155:0/1541130107' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T12:52:16.771 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:16 smithi043 ceph-mon[30062]: from='mgr.? v1:172.21.15.155:0/1541130107' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-21T12:52:16.771 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:16 smithi043 ceph-mon[30062]: from='mgr.? v1:172.21.15.155:0/1541130107' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T12:52:16.771 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:16 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:16.697+0000 7f8bc327d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T12:52:16.772 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:16 smithi043 ceph-mon[41699]: Standby manager daemon x restarted 2024-03-21T12:52:16.772 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:16 smithi043 ceph-mon[41699]: Standby manager daemon x started 2024-03-21T12:52:16.772 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:16 smithi043 ceph-mon[41699]: from='mgr.? v1:172.21.15.155:0/1541130107' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-21T12:52:16.772 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:16 smithi043 ceph-mon[41699]: from='mgr.? v1:172.21.15.155:0/1541130107' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T12:52:16.772 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:16 smithi043 ceph-mon[41699]: from='mgr.? v1:172.21.15.155:0/1541130107' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-21T12:52:16.772 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:16 smithi043 ceph-mon[41699]: from='mgr.? v1:172.21.15.155:0/1541130107' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T12:52:17.030 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:16 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:16.802+0000 7f8bc327d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T12:52:17.426 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:17.029+0000 7f8bc327d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T12:52:17.426 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:52:17] ENGINE Bus STARTING 2024-03-21T12:52:17.426 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: CherryPy Checker: 2024-03-21T12:52:17.426 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: The Application mounted at '' has an empty config. 2024-03-21T12:52:17.427 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T12:52:17.427 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:52:17] ENGINE Serving on http://:::9283 2024-03-21T12:52:17.427 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:52:17] ENGINE Bus STARTED 2024-03-21T12:52:17.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: mgrmap e18: y(active, since 5m), standbys: x 2024-03-21T12:52:17.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: Active manager daemon y restarted 2024-03-21T12:52:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: Activating manager daemon y 2024-03-21T12:52:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: osdmap e73: 8 total, 8 up, 8 in 2024-03-21T12:52:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: mgrmap e19: y(active, starting, since 0.00846208s), standbys: x 2024-03-21T12:52:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T12:52:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:52:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:52:17.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T12:52:17.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T12:52:17.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T12:52:17.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T12:52:17.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T12:52:17.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T12:52:17.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T12:52:17.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T12:52:17.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T12:52:17.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T12:52:17.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T12:52:17.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T12:52:17.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T12:52:17.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: Manager daemon y is now available 2024-03-21T12:52:17.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:52:17.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:17.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:52:17.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-21T12:52:17.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-21T12:52:17.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-21T12:52:17.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-21T12:52:17.738 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: mgrmap e18: y(active, since 5m), standbys: x 2024-03-21T12:52:17.738 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: Active manager daemon y restarted 2024-03-21T12:52:17.738 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: Activating manager daemon y 2024-03-21T12:52:17.738 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: osdmap e73: 8 total, 8 up, 8 in 2024-03-21T12:52:17.738 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: mgrmap e19: y(active, starting, since 0.00846208s), standbys: x 2024-03-21T12:52:17.738 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T12:52:17.738 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:52:17.738 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:52:17.739 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T12:52:17.739 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T12:52:17.739 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T12:52:17.739 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T12:52:17.739 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T12:52:17.739 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T12:52:17.739 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T12:52:17.739 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T12:52:17.739 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T12:52:17.739 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T12:52:17.739 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T12:52:17.739 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T12:52:17.739 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T12:52:17.739 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: Manager daemon y is now available 2024-03-21T12:52:17.740 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:52:17.740 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:17.740 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:52:17.740 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-21T12:52:17.740 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-21T12:52:17.740 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-21T12:52:17.740 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:17 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-21T12:52:17.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: mgrmap e18: y(active, since 5m), standbys: x 2024-03-21T12:52:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: Active manager daemon y restarted 2024-03-21T12:52:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: Activating manager daemon y 2024-03-21T12:52:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: osdmap e73: 8 total, 8 up, 8 in 2024-03-21T12:52:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: mgrmap e19: y(active, starting, since 0.00846208s), standbys: x 2024-03-21T12:52:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T12:52:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T12:52:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T12:52:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T12:52:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T12:52:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T12:52:17.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T12:52:17.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T12:52:17.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T12:52:17.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T12:52:17.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T12:52:17.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T12:52:17.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T12:52:17.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T12:52:17.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T12:52:17.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T12:52:17.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: Manager daemon y is now available 2024-03-21T12:52:17.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:52:17.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:17.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:52:17.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-21T12:52:17.960 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-21T12:52:17.960 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-21T12:52:17.960 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:17 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-21T12:52:18.304 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:52:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:52:18.336 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:52:18 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:52:19.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:19 smithi043 ceph-mon[41699]: [21/Mar/2024:12:52:17] ENGINE Bus STARTING 2024-03-21T12:52:19.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:19 smithi043 ceph-mon[41699]: [21/Mar/2024:12:52:17] ENGINE Serving on https://172.21.15.43:7150 2024-03-21T12:52:19.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:19 smithi043 ceph-mon[41699]: [21/Mar/2024:12:52:18] ENGINE Serving on http://172.21.15.43:8765 2024-03-21T12:52:19.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:19 smithi043 ceph-mon[41699]: [21/Mar/2024:12:52:18] ENGINE Bus STARTED 2024-03-21T12:52:19.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:19 smithi043 ceph-mon[41699]: mgrmap e20: y(active, since 1.01192s), standbys: x 2024-03-21T12:52:19.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:19 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:52:19.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:19 smithi043 ceph-mon[41699]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:52:19.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:19 smithi043 ceph-mon[30062]: [21/Mar/2024:12:52:17] ENGINE Bus STARTING 2024-03-21T12:52:19.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:19 smithi043 ceph-mon[30062]: [21/Mar/2024:12:52:17] ENGINE Serving on https://172.21.15.43:7150 2024-03-21T12:52:19.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:19 smithi043 ceph-mon[30062]: [21/Mar/2024:12:52:18] ENGINE Serving on http://172.21.15.43:8765 2024-03-21T12:52:19.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:19 smithi043 ceph-mon[30062]: [21/Mar/2024:12:52:18] ENGINE Bus STARTED 2024-03-21T12:52:19.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:19 smithi043 ceph-mon[30062]: mgrmap e20: y(active, since 1.01192s), standbys: x 2024-03-21T12:52:19.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:19 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:52:19.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:19 smithi043 ceph-mon[30062]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:52:19.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:19 smithi155 ceph-mon[33846]: [21/Mar/2024:12:52:17] ENGINE Bus STARTING 2024-03-21T12:52:19.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:19 smithi155 ceph-mon[33846]: [21/Mar/2024:12:52:17] ENGINE Serving on https://172.21.15.43:7150 2024-03-21T12:52:19.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:19 smithi155 ceph-mon[33846]: [21/Mar/2024:12:52:18] ENGINE Serving on http://172.21.15.43:8765 2024-03-21T12:52:19.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:19 smithi155 ceph-mon[33846]: [21/Mar/2024:12:52:18] ENGINE Bus STARTED 2024-03-21T12:52:19.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:19 smithi155 ceph-mon[33846]: mgrmap e20: y(active, since 1.01192s), standbys: x 2024-03-21T12:52:19.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:19 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:52:19.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:19 smithi155 ceph-mon[33846]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:52:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:20 smithi155 ceph-mon[33846]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:52:20.222 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:20 smithi043 ceph-mon[30062]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:52:20.222 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:20 smithi043 ceph-mon[41699]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:52:21.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:21 smithi155 ceph-mon[33846]: mgrmap e21: y(active, since 3s), standbys: x 2024-03-21T12:52:21.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:21 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:21.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:21 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:21.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:21 smithi043 ceph-mon[30062]: mgrmap e21: y(active, since 3s), standbys: x 2024-03-21T12:52:21.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:21 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:21.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:21 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:21.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:21 smithi043 ceph-mon[41699]: mgrmap e21: y(active, since 3s), standbys: x 2024-03-21T12:52:21.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:21 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:21.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:21 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:22.668 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[30062]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:52:22.668 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:22.668 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:22.668 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:22.668 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:22.669 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:22.669 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:22.669 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:22.669 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:22.669 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:22.669 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:22.669 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[30062]: Adjusting osd_memory_target on smithi155 to 3760M 2024-03-21T12:52:22.669 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:22.670 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[30062]: mgrmap e22: y(active, since 5s), standbys: x 2024-03-21T12:52:22.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[41699]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:52:22.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:22.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:22.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:22.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:22.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:22.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:22.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:22.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:22.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:22.670 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:22.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[41699]: Adjusting osd_memory_target on smithi155 to 3760M 2024-03-21T12:52:22.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:22.671 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:22 smithi043 ceph-mon[41699]: mgrmap e22: y(active, since 5s), standbys: x 2024-03-21T12:52:22.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:22 smithi155 ceph-mon[33846]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:52:22.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:22 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:22.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:22 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:22.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:22 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:22.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:22 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:22.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:22 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:22.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:22 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:22.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:22 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:22.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:22 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:22.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:22 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:22.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:22 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:22.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:22 smithi155 ceph-mon[33846]: Adjusting osd_memory_target on smithi155 to 3760M 2024-03-21T12:52:22.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:22 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:22.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:22 smithi155 ceph-mon[33846]: mgrmap e22: y(active, since 5s), standbys: x 2024-03-21T12:52:23.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.468+0000 7f50b0f78700 1 -- 172.21.15.43:0/828103352 >> v1:172.21.15.43:6790/0 conn(0x7f50ac10e9d0 legacy=0x7f50ac110e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:23.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.468+0000 7f50b0f78700 1 -- 172.21.15.43:0/828103352 shutdown_connections 2024-03-21T12:52:23.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.468+0000 7f50b0f78700 1 -- 172.21.15.43:0/828103352 >> 172.21.15.43:0/828103352 conn(0x7f50ac0fda70 msgr2=0x7f50ac0ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:23.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.468+0000 7f50b0f78700 1 -- 172.21.15.43:0/828103352 shutdown_connections 2024-03-21T12:52:23.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.468+0000 7f50b0f78700 1 -- 172.21.15.43:0/828103352 wait complete. 2024-03-21T12:52:23.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.469+0000 7f50b0f78700 1 Processor -- start 2024-03-21T12:52:23.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.469+0000 7f50b0f78700 1 -- start start 2024-03-21T12:52:23.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.470+0000 7f50b0f78700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f50ac1a7020 con 0x7f50ac1021a0 2024-03-21T12:52:23.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.470+0000 7f50b0f78700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f50ac1a8240 con 0x7f50ac10ae30 2024-03-21T12:52:23.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.470+0000 7f50b0f78700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f50ac1a9480 con 0x7f50ac10e9d0 2024-03-21T12:52:23.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.470+0000 7f50aa59c700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f50ac1021a0 0x7f50ac1074c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54150/0 (socket says 172.21.15.43:54150) 2024-03-21T12:52:23.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.470+0000 7f50aa59c700 1 -- 172.21.15.43:0/3361059051 learned_addr learned my addr 172.21.15.43:0/3361059051 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:52:23.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.470+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1813117600 0 0) 0x7f50ac1a9480 con 0x7f50ac10e9d0 2024-03-21T12:52:23.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.470+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50980035b0 con 0x7f50ac10e9d0 2024-03-21T12:52:23.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.470+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1691680282 0 0) 0x7f50ac1a7020 con 0x7f50ac1021a0 2024-03-21T12:52:23.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.470+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50ac1a9480 con 0x7f50ac1021a0 2024-03-21T12:52:23.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.470+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 776849050 0 0) 0x7f50ac1a8240 con 0x7f50ac10ae30 2024-03-21T12:52:23.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.470+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50ac1a7020 con 0x7f50ac10ae30 2024-03-21T12:52:23.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.470+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 730189048 0 0) 0x7f50ac1a9480 con 0x7f50ac1021a0 2024-03-21T12:52:23.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.470+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f50ac1a8240 con 0x7f50ac1021a0 2024-03-21T12:52:23.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.470+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2875746751 0 0) 0x7f50980035b0 con 0x7f50ac10e9d0 2024-03-21T12:52:23.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.470+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f50ac1a9480 con 0x7f50ac10e9d0 2024-03-21T12:52:23.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.470+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f509c0013d0 con 0x7f50ac1021a0 2024-03-21T12:52:23.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.470+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f50a40046a0 con 0x7f50ac10e9d0 2024-03-21T12:52:23.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.470+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 863064184 0 0) 0x7f50ac1a7020 con 0x7f50ac10ae30 2024-03-21T12:52:23.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.470+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f50980035b0 con 0x7f50ac10ae30 2024-03-21T12:52:23.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.470+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f5094001830 con 0x7f50ac10ae30 2024-03-21T12:52:23.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.470+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 <== mon.2 v1:172.21.15.43:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3113839885 0 0) 0x7f50ac1a9480 con 0x7f50ac10e9d0 2024-03-21T12:52:23.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.470+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 >> v1:172.21.15.155:6789/0 conn(0x7f50ac10ae30 legacy=0x7f50ac1a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:23.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.470+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 >> v1:172.21.15.43:6789/0 conn(0x7f50ac1021a0 legacy=0x7f50ac1074c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:23.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.470+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 --> v1:172.21.15.43:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ac1aa6c0 con 0x7f50ac10e9d0 2024-03-21T12:52:23.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.471+0000 7f50b0f78700 1 -- 172.21.15.43:0/3361059051 --> v1:172.21.15.43:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ac1a7250 con 0x7f50ac10e9d0 2024-03-21T12:52:23.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.471+0000 7f50b0f78700 1 -- 172.21.15.43:0/3361059051 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ac1a7710 con 0x7f50ac10e9d0 2024-03-21T12:52:23.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.471+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 <== mon.2 v1:172.21.15.43:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f50a4001de0 con 0x7f50ac10e9d0 2024-03-21T12:52:23.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.471+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 <== mon.2 v1:172.21.15.43:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f50a4004f00 con 0x7f50ac10e9d0 2024-03-21T12:52:23.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.471+0000 7f50b0f78700 1 -- 172.21.15.43:0/3361059051 --> v1:172.21.15.43:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f508c000f80 con 0x7f50ac10e9d0 2024-03-21T12:52:23.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.472+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 <== mon.2 v1:172.21.15.43:6790/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7f50a4003a90 con 0x7f50ac10e9d0 2024-03-21T12:52:23.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.473+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 <== mon.2 v1:172.21.15.43:6790/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 915400759 0 0) 0x7f50a4088940 con 0x7f50ac10e9d0 2024-03-21T12:52:23.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.473+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 --> v1:172.21.15.43:6801/3260944688 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f50a4001330 con 0x7f50980798b0 2024-03-21T12:52:23.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.473+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 <== mon.2 v1:172.21.15.43:6790/0 9 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (unknown 3489808268 0 0) 0x7f50a4088ce0 con 0x7f50ac10e9d0 2024-03-21T12:52:23.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.474+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 <== osd.0 v1:172.21.15.43:6801/3260944688 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f50a4001330 con 0x7f50980798b0 2024-03-21T12:52:23.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.490+0000 7f50b0f78700 1 -- 172.21.15.43:0/3361059051 --> v1:172.21.15.43:6801/3260944688 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f508c002ce0 con 0x7f50980798b0 2024-03-21T12:52:23.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.490+0000 7f50a37fe700 1 -- 172.21.15.43:0/3361059051 <== osd.0 v1:172.21.15.43:6801/3260944688 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 1002017476) 0x7f508c002ce0 con 0x7f50980798b0 2024-03-21T12:52:23.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.491+0000 7f50b0f78700 1 -- 172.21.15.43:0/3361059051 >> v1:172.21.15.43:6801/3260944688 conn(0x7f50980798b0 legacy=0x7f509807bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:23.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.491+0000 7f50b0f78700 1 -- 172.21.15.43:0/3361059051 >> v1:172.21.15.43:6800/4199387216 conn(0x7f509806f1d0 legacy=0x7f5098071690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:23.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.491+0000 7f50b0f78700 1 -- 172.21.15.43:0/3361059051 >> v1:172.21.15.43:6790/0 conn(0x7f50ac10e9d0 legacy=0x7f50ac1a56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:23.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.491+0000 7f50aad9d700 1 -- 172.21.15.43:0/3361059051 reap_dead start 2024-03-21T12:52:23.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.491+0000 7f50b0f78700 1 -- 172.21.15.43:0/3361059051 shutdown_connections 2024-03-21T12:52:23.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.491+0000 7f50b0f78700 1 -- 172.21.15.43:0/3361059051 >> 172.21.15.43:0/3361059051 conn(0x7f50ac0fda70 msgr2=0x7f50ac111ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:23.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.491+0000 7f50b0f78700 1 -- 172.21.15.43:0/3361059051 shutdown_connections 2024-03-21T12:52:23.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.491+0000 7f50b0f78700 1 -- 172.21.15.43:0/3361059051 wait complete. 2024-03-21T12:52:23.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.666+0000 7fe8d203f700 1 -- 172.21.15.43:0/142356334 >> v1:172.21.15.43:6790/0 conn(0x7fe8cc10d8b0 legacy=0x7fe8cc10fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:23.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.667+0000 7fe8d203f700 1 -- 172.21.15.43:0/142356334 shutdown_connections 2024-03-21T12:52:23.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.667+0000 7fe8d203f700 1 -- 172.21.15.43:0/142356334 >> 172.21.15.43:0/142356334 conn(0x7fe8cc0fc930 msgr2=0x7fe8cc0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:23.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.667+0000 7fe8d203f700 1 -- 172.21.15.43:0/142356334 shutdown_connections 2024-03-21T12:52:23.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.667+0000 7fe8d203f700 1 -- 172.21.15.43:0/142356334 wait complete. 2024-03-21T12:52:23.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.667+0000 7fe8d203f700 1 Processor -- start 2024-03-21T12:52:23.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.667+0000 7fe8d203f700 1 -- start start 2024-03-21T12:52:23.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.667+0000 7fe8d203f700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe8cc1a91d0 con 0x7fe8cc10d8b0 2024-03-21T12:52:23.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.668+0000 7fe8d203f700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe8cc1aa3f0 con 0x7fe8cc109d10 2024-03-21T12:52:23.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.668+0000 7fe8d203f700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe8cc1ab630 con 0x7fe8cc101080 2024-03-21T12:52:23.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.668+0000 7fe8cb7fe700 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7fe8cc101080 0x7fe8cc0fe7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:52016/0 (socket says 172.21.15.43:52016) 2024-03-21T12:52:23.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.668+0000 7fe8cb7fe700 1 -- 172.21.15.43:0/1708762577 learned_addr learned my addr 172.21.15.43:0/1708762577 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:52:23.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.668+0000 7fe8c8ff9700 1 -- 172.21.15.43:0/1708762577 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2262835920 0 0) 0x7fe8cc1ab630 con 0x7fe8cc101080 2024-03-21T12:52:23.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.668+0000 7fe8c8ff9700 1 -- 172.21.15.43:0/1708762577 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8b40035b0 con 0x7fe8cc101080 2024-03-21T12:52:23.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.668+0000 7fe8c8ff9700 1 -- 172.21.15.43:0/1708762577 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2903246911 0 0) 0x7fe8cc1a91d0 con 0x7fe8cc10d8b0 2024-03-21T12:52:23.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.668+0000 7fe8c8ff9700 1 -- 172.21.15.43:0/1708762577 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8cc1ab630 con 0x7fe8cc10d8b0 2024-03-21T12:52:23.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.668+0000 7fe8c8ff9700 1 -- 172.21.15.43:0/1708762577 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2719903125 0 0) 0x7fe8b40035b0 con 0x7fe8cc101080 2024-03-21T12:52:23.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.668+0000 7fe8c8ff9700 1 -- 172.21.15.43:0/1708762577 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe8cc1a91d0 con 0x7fe8cc101080 2024-03-21T12:52:23.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.668+0000 7fe8c8ff9700 1 -- 172.21.15.43:0/1708762577 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fe8bc0018f0 con 0x7fe8cc101080 2024-03-21T12:52:23.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.668+0000 7fe8c8ff9700 1 -- 172.21.15.43:0/1708762577 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1732362363 0 0) 0x7fe8cc1aa3f0 con 0x7fe8cc109d10 2024-03-21T12:52:23.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.668+0000 7fe8c8ff9700 1 -- 172.21.15.43:0/1708762577 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8b40035b0 con 0x7fe8cc109d10 2024-03-21T12:52:23.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.668+0000 7fe8c8ff9700 1 -- 172.21.15.43:0/1708762577 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2407778783 0 0) 0x7fe8cc1ab630 con 0x7fe8cc10d8b0 2024-03-21T12:52:23.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.668+0000 7fe8c8ff9700 1 -- 172.21.15.43:0/1708762577 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe8cc1aa3f0 con 0x7fe8cc10d8b0 2024-03-21T12:52:23.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.668+0000 7fe8c8ff9700 1 -- 172.21.15.43:0/1708762577 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fe8c4001bc0 con 0x7fe8cc10d8b0 2024-03-21T12:52:23.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.668+0000 7fe8c8ff9700 1 -- 172.21.15.43:0/1708762577 <== mon.2 v1:172.21.15.43:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1280605117 0 0) 0x7fe8cc1a91d0 con 0x7fe8cc101080 2024-03-21T12:52:23.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.668+0000 7fe8c8ff9700 1 -- 172.21.15.43:0/1708762577 >> v1:172.21.15.155:6789/0 conn(0x7fe8cc109d10 legacy=0x7fe8cc1a4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:23.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.668+0000 7fe8c8ff9700 1 -- 172.21.15.43:0/1708762577 >> v1:172.21.15.43:6789/0 conn(0x7fe8cc10d8b0 legacy=0x7fe8cc1a7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:23.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.668+0000 7fe8c8ff9700 1 -- 172.21.15.43:0/1708762577 --> v1:172.21.15.43:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8cc1ac870 con 0x7fe8cc101080 2024-03-21T12:52:23.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.668+0000 7fe8d203f700 1 -- 172.21.15.43:0/1708762577 --> v1:172.21.15.43:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8cc1a9400 con 0x7fe8cc101080 2024-03-21T12:52:23.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.668+0000 7fe8d203f700 1 -- 172.21.15.43:0/1708762577 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8cc1a98c0 con 0x7fe8cc101080 2024-03-21T12:52:23.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.668+0000 7fe8c8ff9700 1 -- 172.21.15.43:0/1708762577 <== mon.2 v1:172.21.15.43:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7fe8bc003b30 con 0x7fe8cc101080 2024-03-21T12:52:23.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.669+0000 7fe8c8ff9700 1 -- 172.21.15.43:0/1708762577 <== mon.2 v1:172.21.15.43:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fe8bc005cf0 con 0x7fe8cc101080 2024-03-21T12:52:23.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.669+0000 7fe8d203f700 1 -- 172.21.15.43:0/1708762577 --> v1:172.21.15.43:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe8b8000f80 con 0x7fe8cc101080 2024-03-21T12:52:23.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.670+0000 7fe8c8ff9700 1 -- 172.21.15.43:0/1708762577 <== mon.2 v1:172.21.15.43:6790/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7fe8bc0036a0 con 0x7fe8cc101080 2024-03-21T12:52:23.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.670+0000 7fe8c8ff9700 1 -- 172.21.15.43:0/1708762577 <== mon.2 v1:172.21.15.43:6790/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 915400759 0 0) 0x7fe8bc089a20 con 0x7fe8cc101080 2024-03-21T12:52:23.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.670+0000 7fe8c8ff9700 1 -- 172.21.15.43:0/1708762577 --> v1:172.21.15.43:6805/1862805780 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe8c4001400 con 0x7fe8b4079a60 2024-03-21T12:52:23.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.670+0000 7fe8c8ff9700 1 -- 172.21.15.43:0/1708762577 <== mon.2 v1:172.21.15.43:6790/0 9 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (unknown 3489808268 0 0) 0x7fe8bc089e70 con 0x7fe8cc101080 2024-03-21T12:52:23.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.671+0000 7fe8c8ff9700 1 -- 172.21.15.43:0/1708762577 <== osd.1 v1:172.21.15.43:6805/1862805780 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fe8c4001400 con 0x7fe8b4079a60 2024-03-21T12:52:23.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.688+0000 7fe8d203f700 1 -- 172.21.15.43:0/1708762577 --> v1:172.21.15.43:6805/1862805780 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe8b8002ce0 con 0x7fe8b4079a60 2024-03-21T12:52:23.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.688+0000 7fe8c8ff9700 1 -- 172.21.15.43:0/1708762577 <== osd.1 v1:172.21.15.43:6805/1862805780 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 4200525082) 0x7fe8b8002ce0 con 0x7fe8b4079a60 2024-03-21T12:52:23.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.688+0000 7fe8d203f700 1 -- 172.21.15.43:0/1708762577 >> v1:172.21.15.43:6805/1862805780 conn(0x7fe8b4079a60 legacy=0x7fe8b407bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:23.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.688+0000 7fe8d203f700 1 -- 172.21.15.43:0/1708762577 >> v1:172.21.15.43:6800/4199387216 conn(0x7fe8b406f300 legacy=0x7fe8b40717c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:23.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.688+0000 7fe8d203f700 1 -- 172.21.15.43:0/1708762577 >> v1:172.21.15.43:6790/0 conn(0x7fe8cc101080 legacy=0x7fe8cc0fe7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:23.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.688+0000 7fe8cbfff700 1 -- 172.21.15.43:0/1708762577 reap_dead start 2024-03-21T12:52:23.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.689+0000 7fe8d203f700 1 -- 172.21.15.43:0/1708762577 shutdown_connections 2024-03-21T12:52:23.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.689+0000 7fe8d203f700 1 -- 172.21.15.43:0/1708762577 >> 172.21.15.43:0/1708762577 conn(0x7fe8cc0fc930 msgr2=0x7fe8cc110520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:23.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.689+0000 7fe8d203f700 1 -- 172.21.15.43:0/1708762577 shutdown_connections 2024-03-21T12:52:23.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:23.689+0000 7fe8d203f700 1 -- 172.21.15.43:0/1708762577 wait complete. 2024-03-21T12:52:24.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.214+0000 7fe03988a700 1 -- 172.21.15.43:0/3901362084 >> v1:172.21.15.43:6789/0 conn(0x7fe03410ae30 legacy=0x7fe03410d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:24.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.214+0000 7fe03988a700 1 -- 172.21.15.43:0/3901362084 shutdown_connections 2024-03-21T12:52:24.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.214+0000 7fe03988a700 1 -- 172.21.15.43:0/3901362084 >> 172.21.15.43:0/3901362084 conn(0x7fe0340fda70 msgr2=0x7fe0340ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:24.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.214+0000 7fe03988a700 1 -- 172.21.15.43:0/3901362084 shutdown_connections 2024-03-21T12:52:24.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.214+0000 7fe03988a700 1 -- 172.21.15.43:0/3901362084 wait complete. 2024-03-21T12:52:24.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.215+0000 7fe03988a700 1 Processor -- start 2024-03-21T12:52:24.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.215+0000 7fe03988a700 1 -- start start 2024-03-21T12:52:24.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.215+0000 7fe03988a700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe0341a6df0 con 0x7fe0341021a0 2024-03-21T12:52:24.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.215+0000 7fe03988a700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe0341a8010 con 0x7fe03410ae30 2024-03-21T12:52:24.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.215+0000 7fe03988a700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe0341a9230 con 0x7fe03410e9d0 2024-03-21T12:52:24.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.215+0000 7fe032ffd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe0341021a0 0x7fe034107270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54166/0 (socket says 172.21.15.43:54166) 2024-03-21T12:52:24.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.215+0000 7fe032ffd700 1 -- 172.21.15.43:0/2557801272 learned_addr learned my addr 172.21.15.43:0/2557801272 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:52:24.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.215+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2615945460 0 0) 0x7fe0341a6df0 con 0x7fe0341021a0 2024-03-21T12:52:24.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.215+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe01c0035b0 con 0x7fe0341021a0 2024-03-21T12:52:24.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.215+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2713392565 0 0) 0x7fe0341a9230 con 0x7fe03410e9d0 2024-03-21T12:52:24.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.215+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0341a6df0 con 0x7fe03410e9d0 2024-03-21T12:52:24.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.216+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2875466550 0 0) 0x7fe0341a8010 con 0x7fe03410ae30 2024-03-21T12:52:24.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.216+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0341a9230 con 0x7fe03410ae30 2024-03-21T12:52:24.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.216+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 11966655 0 0) 0x7fe01c0035b0 con 0x7fe0341021a0 2024-03-21T12:52:24.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.216+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe0341a8010 con 0x7fe0341021a0 2024-03-21T12:52:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.216+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fe0240019c0 con 0x7fe0341021a0 2024-03-21T12:52:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.216+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1352311005 0 0) 0x7fe0341a6df0 con 0x7fe03410e9d0 2024-03-21T12:52:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.216+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe01c0035b0 con 0x7fe03410e9d0 2024-03-21T12:52:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.216+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fe02c001540 con 0x7fe03410e9d0 2024-03-21T12:52:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.216+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 114688443 0 0) 0x7fe0341a9230 con 0x7fe03410ae30 2024-03-21T12:52:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.216+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe0341a6df0 con 0x7fe03410ae30 2024-03-21T12:52:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.216+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fe028004510 con 0x7fe03410ae30 2024-03-21T12:52:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.216+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 <== mon.2 v1:172.21.15.43:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 440312124 0 0) 0x7fe01c0035b0 con 0x7fe03410e9d0 2024-03-21T12:52:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.216+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 >> v1:172.21.15.155:6789/0 conn(0x7fe03410ae30 legacy=0x7fe0341a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.216+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 >> v1:172.21.15.43:6789/0 conn(0x7fe0341021a0 legacy=0x7fe034107270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.216+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 --> v1:172.21.15.43:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0341aa470 con 0x7fe03410e9d0 2024-03-21T12:52:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.216+0000 7fe03988a700 1 -- 172.21.15.43:0/2557801272 --> v1:172.21.15.43:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0341a9460 con 0x7fe03410e9d0 2024-03-21T12:52:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.216+0000 7fe03988a700 1 -- 172.21.15.43:0/2557801272 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0341a9920 con 0x7fe03410e9d0 2024-03-21T12:52:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.216+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 <== mon.2 v1:172.21.15.43:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7fe02c003e90 con 0x7fe03410e9d0 2024-03-21T12:52:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.216+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 <== mon.2 v1:172.21.15.43:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fe02c004b90 con 0x7fe03410e9d0 2024-03-21T12:52:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.217+0000 7fe03988a700 1 -- 172.21.15.43:0/2557801272 --> v1:172.21.15.43:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe020000f80 con 0x7fe03410e9d0 2024-03-21T12:52:24.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.218+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 <== mon.2 v1:172.21.15.43:6790/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7fe02c003a00 con 0x7fe03410e9d0 2024-03-21T12:52:24.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.218+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 <== mon.2 v1:172.21.15.43:6790/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 915400759 0 0) 0x7fe02c088500 con 0x7fe03410e9d0 2024-03-21T12:52:24.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.218+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 --> v1:172.21.15.155:6800/2961807269 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe01c07d890 con 0x7fe01c079b80 2024-03-21T12:52:24.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.218+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 <== mon.2 v1:172.21.15.43:6790/0 9 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (unknown 3489808268 0 0) 0x7fe02c088950 con 0x7fe03410e9d0 2024-03-21T12:52:24.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.220+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 <== osd.4 v1:172.21.15.155:6800/2961807269 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fe01c07d890 con 0x7fe01c079b80 2024-03-21T12:52:24.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.236+0000 7fe03988a700 1 -- 172.21.15.43:0/2557801272 --> v1:172.21.15.155:6800/2961807269 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe020002ce0 con 0x7fe01c079b80 2024-03-21T12:52:24.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.237+0000 7fe038888700 1 -- 172.21.15.43:0/2557801272 <== osd.4 v1:172.21.15.155:6800/2961807269 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 3087939264) 0x7fe020002ce0 con 0x7fe01c079b80 2024-03-21T12:52:24.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.237+0000 7fe03988a700 1 -- 172.21.15.43:0/2557801272 >> v1:172.21.15.155:6800/2961807269 conn(0x7fe01c079b80 legacy=0x7fe01c07c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:24.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.237+0000 7fe03988a700 1 -- 172.21.15.43:0/2557801272 >> v1:172.21.15.43:6800/4199387216 conn(0x7fe01c06f2c0 legacy=0x7fe01c071780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:24.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.237+0000 7fe03988a700 1 -- 172.21.15.43:0/2557801272 >> v1:172.21.15.43:6790/0 conn(0x7fe03410e9d0 legacy=0x7fe0341a5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:24.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.237+0000 7fe0337fe700 1 -- 172.21.15.43:0/2557801272 reap_dead start 2024-03-21T12:52:24.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.237+0000 7fe03988a700 1 -- 172.21.15.43:0/2557801272 shutdown_connections 2024-03-21T12:52:24.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.237+0000 7fe03988a700 1 -- 172.21.15.43:0/2557801272 >> 172.21.15.43:0/2557801272 conn(0x7fe0340fda70 msgr2=0x7fe0340fde50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:24.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.237+0000 7fe03988a700 1 -- 172.21.15.43:0/2557801272 shutdown_connections 2024-03-21T12:52:24.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:24.237+0000 7fe03988a700 1 -- 172.21.15.43:0/2557801272 wait complete. 2024-03-21T12:52:24.555 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:24 smithi043 ceph-mon[30062]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:52:24.556 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:24 smithi043 ceph-mon[41699]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:52:24.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:24 smithi155 ceph-mon[33846]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:52:25.228 INFO:teuthology.orchestra.run.smithi043.stdout:133143986206 2024-03-21T12:52:25.228 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-21T12:52:26.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.108+0000 7f944102b700 1 -- 172.21.15.43:0/3035677172 >> v1:172.21.15.43:6790/0 conn(0x7f943c1021a0 legacy=0x7f943c102580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:26.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.108+0000 7f944102b700 1 -- 172.21.15.43:0/3035677172 shutdown_connections 2024-03-21T12:52:26.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.108+0000 7f944102b700 1 -- 172.21.15.43:0/3035677172 >> 172.21.15.43:0/3035677172 conn(0x7f943c0fda70 msgr2=0x7f943c0ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:26.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.108+0000 7f944102b700 1 -- 172.21.15.43:0/3035677172 shutdown_connections 2024-03-21T12:52:26.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.108+0000 7f944102b700 1 -- 172.21.15.43:0/3035677172 wait complete. 2024-03-21T12:52:26.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.109+0000 7f944102b700 1 Processor -- start 2024-03-21T12:52:26.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.109+0000 7f944102b700 1 -- start start 2024-03-21T12:52:26.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.109+0000 7f944102b700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f943c1a6f30 con 0x7f943c1021a0 2024-03-21T12:52:26.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.109+0000 7f944102b700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f943c1a8150 con 0x7f943c10ae30 2024-03-21T12:52:26.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.109+0000 7f944102b700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f943c1a9390 con 0x7f943c10e9d0 2024-03-21T12:52:26.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.109+0000 7f943ad9d700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f943c1021a0 0x7f943c1073d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54180/0 (socket says 172.21.15.43:54180) 2024-03-21T12:52:26.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.110+0000 7f943ad9d700 1 -- 172.21.15.43:0/2877477869 learned_addr learned my addr 172.21.15.43:0/2877477869 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:52:26.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.110+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2760176715 0 0) 0x7f943c1a6f30 con 0x7f943c1021a0 2024-03-21T12:52:26.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.110+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94240031f0 con 0x7f943c1021a0 2024-03-21T12:52:26.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.110+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 218761911 0 0) 0x7f943c1a9390 con 0x7f943c10e9d0 2024-03-21T12:52:26.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.110+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f943c1a6f30 con 0x7f943c10e9d0 2024-03-21T12:52:26.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.110+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 830391627 0 0) 0x7f943c1a8150 con 0x7f943c10ae30 2024-03-21T12:52:26.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.110+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f943c1a9390 con 0x7f943c10ae30 2024-03-21T12:52:26.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.110+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 707166583 0 0) 0x7f94240031f0 con 0x7f943c1021a0 2024-03-21T12:52:26.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.110+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f943c1a8150 con 0x7f943c1021a0 2024-03-21T12:52:26.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.110+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f942c0044b0 con 0x7f943c1021a0 2024-03-21T12:52:26.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.110+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 341935544 0 0) 0x7f943c1a9390 con 0x7f943c10ae30 2024-03-21T12:52:26.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.110+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f94240031f0 con 0x7f943c10ae30 2024-03-21T12:52:26.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.110+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f9430001000 con 0x7f943c10ae30 2024-03-21T12:52:26.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.110+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2758802274 0 0) 0x7f943c1a6f30 con 0x7f943c10e9d0 2024-03-21T12:52:26.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.110+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f943c1a9390 con 0x7f943c10e9d0 2024-03-21T12:52:26.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.110+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f9434001af0 con 0x7f943c10e9d0 2024-03-21T12:52:26.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.110+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2628485175 0 0) 0x7f94240031f0 con 0x7f943c10ae30 2024-03-21T12:52:26.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.110+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 >> v1:172.21.15.43:6790/0 conn(0x7f943c10e9d0 legacy=0x7f943c1a55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:26.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.110+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 >> v1:172.21.15.43:6789/0 conn(0x7f943c1021a0 legacy=0x7f943c1073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:26.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.110+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f943c1aa5d0 con 0x7f943c10ae30 2024-03-21T12:52:26.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.111+0000 7f944102b700 1 -- 172.21.15.43:0/2877477869 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f943c1a8380 con 0x7f943c10ae30 2024-03-21T12:52:26.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.111+0000 7f944102b700 1 -- 172.21.15.43:0/2877477869 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f943c1a8840 con 0x7f943c10ae30 2024-03-21T12:52:26.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.111+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f9430001620 con 0x7f943c10ae30 2024-03-21T12:52:26.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.111+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f94300048d0 con 0x7f943c10ae30 2024-03-21T12:52:26.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.112+0000 7f944102b700 1 -- 172.21.15.43:0/2877477869 --> v1:172.21.15.155:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9428000f80 con 0x7f943c10ae30 2024-03-21T12:52:26.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.112+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7f9430004be0 con 0x7f943c10ae30 2024-03-21T12:52:26.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.113+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 915400759 0 0) 0x7f9430088420 con 0x7f943c10ae30 2024-03-21T12:52:26.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.113+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 --> v1:172.21.15.155:6812/226070080 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f942407d650 con 0x7f9424079940 2024-03-21T12:52:26.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.113+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (unknown 3489808268 0 0) 0x7f9430088830 con 0x7f943c10ae30 2024-03-21T12:52:26.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.114+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 <== osd.7 v1:172.21.15.155:6812/226070080 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f942407d650 con 0x7f9424079940 2024-03-21T12:52:26.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.133+0000 7f944102b700 1 -- 172.21.15.43:0/2877477869 --> v1:172.21.15.155:6812/226070080 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9428002ce0 con 0x7f9424079940 2024-03-21T12:52:26.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.133+0000 7f9423fff700 1 -- 172.21.15.43:0/2877477869 <== osd.7 v1:172.21.15.155:6812/226070080 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 1312652062) 0x7f9428002ce0 con 0x7f9424079940 2024-03-21T12:52:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.133+0000 7f944102b700 1 -- 172.21.15.43:0/2877477869 >> v1:172.21.15.155:6812/226070080 conn(0x7f9424079940 legacy=0x7f942407bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.133+0000 7f944102b700 1 -- 172.21.15.43:0/2877477869 >> v1:172.21.15.43:6800/4199387216 conn(0x7f942406f260 legacy=0x7f9424071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.133+0000 7f944102b700 1 -- 172.21.15.43:0/2877477869 >> v1:172.21.15.155:6789/0 conn(0x7f943c10ae30 legacy=0x7f943c1a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.133+0000 7f943b59e700 1 -- 172.21.15.43:0/2877477869 reap_dead start 2024-03-21T12:52:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.133+0000 7f944102b700 1 -- 172.21.15.43:0/2877477869 shutdown_connections 2024-03-21T12:52:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.134+0000 7f944102b700 1 -- 172.21.15.43:0/2877477869 >> 172.21.15.43:0/2877477869 conn(0x7f943c0fda70 msgr2=0x7f943c10ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.134+0000 7f944102b700 1 -- 172.21.15.43:0/2877477869 shutdown_connections 2024-03-21T12:52:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:26.134+0000 7f944102b700 1 -- 172.21.15.43:0/2877477869 wait complete. 2024-03-21T12:52:26.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:26 smithi155 ceph-mon[33846]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-21T12:52:26.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:26 smithi043 ceph-mon[30062]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-21T12:52:26.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:26 smithi043 ceph-mon[41699]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-21T12:52:27.088 INFO:teuthology.orchestra.run.smithi043.stdout:34359738418 2024-03-21T12:52:27.088 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-21T12:52:28.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:52:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:52:28.293 INFO:teuthology.orchestra.run.smithi043.stdout:55834574892 2024-03-21T12:52:28.293 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-21T12:52:28.417 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:52:28 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:52:28.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.494+0000 7f4cbc129700 1 -- 172.21.15.43:0/1608416725 >> v1:172.21.15.43:6789/0 conn(0x7f4cb4101080 legacy=0x7f4cb4101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:28.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.494+0000 7f4cbc129700 1 -- 172.21.15.43:0/1608416725 shutdown_connections 2024-03-21T12:52:28.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.494+0000 7f4cbc129700 1 -- 172.21.15.43:0/1608416725 >> 172.21.15.43:0/1608416725 conn(0x7f4cb40fc930 msgr2=0x7f4cb40fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:28.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.494+0000 7f4cbc129700 1 -- 172.21.15.43:0/1608416725 shutdown_connections 2024-03-21T12:52:28.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.494+0000 7f4cbc129700 1 -- 172.21.15.43:0/1608416725 wait complete. 2024-03-21T12:52:28.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.494+0000 7f4cbc129700 1 Processor -- start 2024-03-21T12:52:28.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.494+0000 7f4cbc129700 1 -- start start 2024-03-21T12:52:28.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.495+0000 7f4cbc129700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4cb41a7040 con 0x7f4cb4109d10 2024-03-21T12:52:28.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.495+0000 7f4cbc129700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4cb41a8260 con 0x7f4cb4101080 2024-03-21T12:52:28.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.495+0000 7f4cbc129700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4cb41a94a0 con 0x7f4cb410d8b0 2024-03-21T12:52:28.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.495+0000 7f4cba6c6700 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7f4cb410d8b0 0x7f4cb41a5700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:52066/0 (socket says 172.21.15.43:52066) 2024-03-21T12:52:28.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.495+0000 7f4cba6c6700 1 -- 172.21.15.43:0/394864741 learned_addr learned my addr 172.21.15.43:0/394864741 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:52:28.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.495+0000 7f4cb96c4700 1 --1- 172.21.15.43:0/394864741 >> v1:172.21.15.43:6789/0 conn(0x7f4cb4109d10 0x7f4cb41a1d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54192/0 (socket says 172.21.15.43:54192) 2024-03-21T12:52:28.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.495+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302527640 0 0) 0x7f4cb41a7040 con 0x7f4cb4109d10 2024-03-21T12:52:28.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.495+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ca00035b0 con 0x7f4cb4109d10 2024-03-21T12:52:28.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.495+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581162081 0 0) 0x7f4cb41a94a0 con 0x7f4cb410d8b0 2024-03-21T12:52:28.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.495+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cb41a7040 con 0x7f4cb410d8b0 2024-03-21T12:52:28.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.495+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 277291586 0 0) 0x7f4cb41a8260 con 0x7f4cb4101080 2024-03-21T12:52:28.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.495+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cb41a94a0 con 0x7f4cb4101080 2024-03-21T12:52:28.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.495+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1655705813 0 0) 0x7f4ca00035b0 con 0x7f4cb4109d10 2024-03-21T12:52:28.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.495+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4cb41a8260 con 0x7f4cb4109d10 2024-03-21T12:52:28.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.495+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f4cb0001a30 con 0x7f4cb4109d10 2024-03-21T12:52:28.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.495+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1740449287 0 0) 0x7f4cb41a7040 con 0x7f4cb410d8b0 2024-03-21T12:52:28.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.495+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4ca00035b0 con 0x7f4cb410d8b0 2024-03-21T12:52:28.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.495+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f4cac001bb0 con 0x7f4cb410d8b0 2024-03-21T12:52:28.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.495+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1593587069 0 0) 0x7f4cb41a94a0 con 0x7f4cb4101080 2024-03-21T12:52:28.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.496+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4cb41a7040 con 0x7f4cb4101080 2024-03-21T12:52:28.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.496+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f4ca4004410 con 0x7f4cb4101080 2024-03-21T12:52:28.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.496+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2648981979 0 0) 0x7f4cb41a8260 con 0x7f4cb4109d10 2024-03-21T12:52:28.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.496+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 >> v1:172.21.15.43:6790/0 conn(0x7f4cb410d8b0 legacy=0x7f4cb41a5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:28.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.496+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 >> v1:172.21.15.155:6789/0 conn(0x7f4cb4101080 legacy=0x7f4cb4106410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:28.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.496+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb41aa6e0 con 0x7f4cb4109d10 2024-03-21T12:52:28.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.496+0000 7f4cbc129700 1 -- 172.21.15.43:0/394864741 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb41a96d0 con 0x7f4cb4109d10 2024-03-21T12:52:28.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.496+0000 7f4cbc129700 1 -- 172.21.15.43:0/394864741 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb41a9b90 con 0x7f4cb4109d10 2024-03-21T12:52:28.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.496+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f4cb0001590 con 0x7f4cb4109d10 2024-03-21T12:52:28.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.496+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f4cb0005c30 con 0x7f4cb4109d10 2024-03-21T12:52:28.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.497+0000 7f4cbc129700 1 -- 172.21.15.43:0/394864741 --> v1:172.21.15.43:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4c98000f80 con 0x7f4cb4109d10 2024-03-21T12:52:28.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.497+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7f4cb0003930 con 0x7f4cb4109d10 2024-03-21T12:52:28.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.498+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 915400759 0 0) 0x7f4cb0089610 con 0x7f4cb4109d10 2024-03-21T12:52:28.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.498+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 --> v1:172.21.15.155:6804/1838740519 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ca007d580 con 0x7f4ca0079870 2024-03-21T12:52:28.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.498+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (unknown 3489808268 0 0) 0x7f4cb0089a60 con 0x7f4cb4109d10 2024-03-21T12:52:28.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.500+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 <== osd.5 v1:172.21.15.155:6804/1838740519 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f4ca007d580 con 0x7f4ca0079870 2024-03-21T12:52:28.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.518+0000 7f4cbc129700 1 -- 172.21.15.43:0/394864741 --> v1:172.21.15.155:6804/1838740519 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4c98002ce0 con 0x7f4ca0079870 2024-03-21T12:52:28.518 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:28 smithi043 ceph-mon[30062]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-21T12:52:28.519 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:28 smithi043 ceph-mon[41699]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-21T12:52:28.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.519+0000 7f4caaffd700 1 -- 172.21.15.43:0/394864741 <== osd.5 v1:172.21.15.155:6804/1838740519 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 2394879001) 0x7f4c98002ce0 con 0x7f4ca0079870 2024-03-21T12:52:28.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.519+0000 7f4cbc129700 1 -- 172.21.15.43:0/394864741 >> v1:172.21.15.155:6804/1838740519 conn(0x7f4ca0079870 legacy=0x7f4ca007bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:28.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.519+0000 7f4cbc129700 1 -- 172.21.15.43:0/394864741 >> v1:172.21.15.43:6800/4199387216 conn(0x7f4ca006f190 legacy=0x7f4ca0071650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:28.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.519+0000 7f4cbc129700 1 -- 172.21.15.43:0/394864741 >> v1:172.21.15.43:6789/0 conn(0x7f4cb4109d10 legacy=0x7f4cb41a1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:28.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.519+0000 7f4cba6c6700 1 -- 172.21.15.43:0/394864741 reap_dead start 2024-03-21T12:52:28.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.519+0000 7f4cbc129700 1 -- 172.21.15.43:0/394864741 shutdown_connections 2024-03-21T12:52:28.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.519+0000 7f4cbc129700 1 -- 172.21.15.43:0/394864741 >> 172.21.15.43:0/394864741 conn(0x7f4cb40fc930 msgr2=0x7f4cb406ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:28.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.520+0000 7f4cbc129700 1 -- 172.21.15.43:0/394864741 shutdown_connections 2024-03-21T12:52:28.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:28.520+0000 7f4cbc129700 1 -- 172.21.15.43:0/394864741 wait complete. 2024-03-21T12:52:28.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:28 smithi155 ceph-mon[33846]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-21T12:52:29.645 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:29 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:52:29.645 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:29 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:52:29.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:29 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:52:29.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.752+0000 7fd62fb76700 1 -- 172.21.15.43:0/1671789879 >> v1:172.21.15.43:6789/0 conn(0x7fd6281021b0 legacy=0x7fd628102590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:29.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.752+0000 7fd62fb76700 1 -- 172.21.15.43:0/1671789879 shutdown_connections 2024-03-21T12:52:29.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.752+0000 7fd62fb76700 1 -- 172.21.15.43:0/1671789879 >> 172.21.15.43:0/1671789879 conn(0x7fd6280fda80 msgr2=0x7fd6280ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:29.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.753+0000 7fd62fb76700 1 -- 172.21.15.43:0/1671789879 shutdown_connections 2024-03-21T12:52:29.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.753+0000 7fd62fb76700 1 -- 172.21.15.43:0/1671789879 wait complete. 2024-03-21T12:52:29.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.753+0000 7fd62fb76700 1 Processor -- start 2024-03-21T12:52:29.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.753+0000 7fd62fb76700 1 -- start start 2024-03-21T12:52:29.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.753+0000 7fd62fb76700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd6281a6e00 con 0x7fd62810e9e0 2024-03-21T12:52:29.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.753+0000 7fd62fb76700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd6281a8020 con 0x7fd6281021b0 2024-03-21T12:52:29.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.753+0000 7fd62fb76700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd6281a9260 con 0x7fd62810ae40 2024-03-21T12:52:29.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.753+0000 7fd62d111700 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7fd62810ae40 0x7fd6281a1e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:52084/0 (socket says 172.21.15.43:52084) 2024-03-21T12:52:29.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.753+0000 7fd62e113700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd62810e9e0 0x7fd6281a5580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54206/0 (socket says 172.21.15.43:54206) 2024-03-21T12:52:29.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.753+0000 7fd62d111700 1 -- 172.21.15.43:0/369047716 learned_addr learned my addr 172.21.15.43:0/369047716 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:52:29.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.754+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3428712833 0 0) 0x7fd6281a6e00 con 0x7fd62810e9e0 2024-03-21T12:52:29.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.754+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6140031f0 con 0x7fd62810e9e0 2024-03-21T12:52:29.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.754+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1314644980 0 0) 0x7fd6281a9260 con 0x7fd62810ae40 2024-03-21T12:52:29.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.754+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6281a6e00 con 0x7fd62810ae40 2024-03-21T12:52:29.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.754+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 706577446 0 0) 0x7fd6281a8020 con 0x7fd6281021b0 2024-03-21T12:52:29.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.754+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6281a9260 con 0x7fd6281021b0 2024-03-21T12:52:29.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.754+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3658814609 0 0) 0x7fd6281a6e00 con 0x7fd62810ae40 2024-03-21T12:52:29.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.754+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd6281a8020 con 0x7fd62810ae40 2024-03-21T12:52:29.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.754+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4252994548 0 0) 0x7fd6140031f0 con 0x7fd62810e9e0 2024-03-21T12:52:29.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.754+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd6281a6e00 con 0x7fd62810e9e0 2024-03-21T12:52:29.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.754+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fd6180019e0 con 0x7fd62810ae40 2024-03-21T12:52:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.754+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fd620001b90 con 0x7fd62810e9e0 2024-03-21T12:52:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.754+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 492200782 0 0) 0x7fd6281a9260 con 0x7fd6281021b0 2024-03-21T12:52:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.754+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd6140031f0 con 0x7fd6281021b0 2024-03-21T12:52:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.754+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fd624001920 con 0x7fd6281021b0 2024-03-21T12:52:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.754+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 <== mon.2 v1:172.21.15.43:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1504397142 0 0) 0x7fd6281a8020 con 0x7fd62810ae40 2024-03-21T12:52:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.754+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 >> v1:172.21.15.155:6789/0 conn(0x7fd6281021b0 legacy=0x7fd628107290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.754+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 >> v1:172.21.15.43:6789/0 conn(0x7fd62810e9e0 legacy=0x7fd6281a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.754+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 --> v1:172.21.15.43:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6281aa4a0 con 0x7fd62810ae40 2024-03-21T12:52:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.754+0000 7fd62fb76700 1 -- 172.21.15.43:0/369047716 --> v1:172.21.15.43:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6281a8250 con 0x7fd62810ae40 2024-03-21T12:52:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.754+0000 7fd62fb76700 1 -- 172.21.15.43:0/369047716 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6281a8710 con 0x7fd62810ae40 2024-03-21T12:52:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.754+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 <== mon.2 v1:172.21.15.43:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7fd618001c70 con 0x7fd62810ae40 2024-03-21T12:52:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.755+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 <== mon.2 v1:172.21.15.43:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fd618005ca0 con 0x7fd62810ae40 2024-03-21T12:52:29.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.755+0000 7fd62fb76700 1 -- 172.21.15.43:0/369047716 --> v1:172.21.15.43:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd60c000f80 con 0x7fd62810ae40 2024-03-21T12:52:29.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.756+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 <== mon.2 v1:172.21.15.43:6790/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7fd618003930 con 0x7fd62810ae40 2024-03-21T12:52:29.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.756+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 <== mon.2 v1:172.21.15.43:6790/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 915400759 0 0) 0x7fd618089700 con 0x7fd62810ae40 2024-03-21T12:52:29.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.756+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 --> v1:172.21.15.43:6813/2246303296 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd61407d5d0 con 0x7fd6140798c0 2024-03-21T12:52:29.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.756+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 <== mon.2 v1:172.21.15.43:6790/0 9 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (unknown 3489808268 0 0) 0x7fd618089b70 con 0x7fd62810ae40 2024-03-21T12:52:29.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.757+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 <== osd.3 v1:172.21.15.43:6813/2246303296 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7fd61407d5d0 con 0x7fd6140798c0 2024-03-21T12:52:29.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.774+0000 7fd62fb76700 1 -- 172.21.15.43:0/369047716 --> v1:172.21.15.43:6813/2246303296 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd60c002ce0 con 0x7fd6140798c0 2024-03-21T12:52:29.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.774+0000 7fd61effd700 1 -- 172.21.15.43:0/369047716 <== osd.3 v1:172.21.15.43:6813/2246303296 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 2851127663) 0x7fd60c002ce0 con 0x7fd6140798c0 2024-03-21T12:52:29.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.774+0000 7fd62fb76700 1 -- 172.21.15.43:0/369047716 >> v1:172.21.15.43:6813/2246303296 conn(0x7fd6140798c0 legacy=0x7fd61407bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:29.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.774+0000 7fd62fb76700 1 -- 172.21.15.43:0/369047716 >> v1:172.21.15.43:6800/4199387216 conn(0x7fd61406f160 legacy=0x7fd614071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:29.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.774+0000 7fd62fb76700 1 -- 172.21.15.43:0/369047716 >> v1:172.21.15.43:6790/0 conn(0x7fd62810ae40 legacy=0x7fd6281a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:29.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.774+0000 7fd62e113700 1 -- 172.21.15.43:0/369047716 reap_dead start 2024-03-21T12:52:29.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.774+0000 7fd62fb76700 1 -- 172.21.15.43:0/369047716 shutdown_connections 2024-03-21T12:52:29.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.774+0000 7fd62fb76700 1 -- 172.21.15.43:0/369047716 >> 172.21.15.43:0/369047716 conn(0x7fd6280fda80 msgr2=0x7fd62810ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:29.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.774+0000 7fd62fb76700 1 -- 172.21.15.43:0/369047716 shutdown_connections 2024-03-21T12:52:29.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:29.774+0000 7fd62fb76700 1 -- 172.21.15.43:0/369047716 wait complete. 2024-03-21T12:52:30.331 INFO:teuthology.orchestra.run.smithi043.stdout:167503724570 2024-03-21T12:52:30.331 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-21T12:52:30.642 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:30 smithi043 ceph-mon[30062]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-21T12:52:30.643 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:30 smithi043 ceph-mon[41699]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-21T12:52:30.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:30 smithi155 ceph-mon[33846]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-21T12:52:31.207 INFO:teuthology.orchestra.run.smithi043.stdout:111669149732 2024-03-21T12:52:31.208 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-21T12:52:31.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.433+0000 7f91b6eea700 1 -- 172.21.15.43:0/3690195702 >> v1:172.21.15.43:6790/0 conn(0x7f91b0101080 legacy=0x7f91b0101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:31.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.433+0000 7f91b6eea700 1 -- 172.21.15.43:0/3690195702 shutdown_connections 2024-03-21T12:52:31.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.433+0000 7f91b6eea700 1 -- 172.21.15.43:0/3690195702 >> 172.21.15.43:0/3690195702 conn(0x7f91b00fc930 msgr2=0x7f91b00fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:31.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.433+0000 7f91b6eea700 1 -- 172.21.15.43:0/3690195702 shutdown_connections 2024-03-21T12:52:31.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.433+0000 7f91b6eea700 1 -- 172.21.15.43:0/3690195702 wait complete. 2024-03-21T12:52:31.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.434+0000 7f91b6eea700 1 Processor -- start 2024-03-21T12:52:31.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.434+0000 7f91b6eea700 1 -- start start 2024-03-21T12:52:31.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.434+0000 7f91b6eea700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f91b006b110 con 0x7f91b010d8b0 2024-03-21T12:52:31.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.434+0000 7f91b6eea700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f91b006b2e0 con 0x7f91b0109d10 2024-03-21T12:52:31.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.434+0000 7f91b6eea700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f91b01a9500 con 0x7f91b0101080 2024-03-21T12:52:31.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.434+0000 7f91b4c86700 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7f91b0101080 0x7f91b01055c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:52116/0 (socket says 172.21.15.43:52116) 2024-03-21T12:52:31.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.434+0000 7f91b4c86700 1 -- 172.21.15.43:0/583242852 learned_addr learned my addr 172.21.15.43:0/583242852 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:52:31.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.434+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2245162428 0 0) 0x7f91b006b2e0 con 0x7f91b0109d10 2024-03-21T12:52:31.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.434+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91a00035b0 con 0x7f91b0109d10 2024-03-21T12:52:31.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.434+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 753654217 0 0) 0x7f91b01a9500 con 0x7f91b0101080 2024-03-21T12:52:31.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.434+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91b006b2e0 con 0x7f91b0101080 2024-03-21T12:52:31.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.434+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 370068985 0 0) 0x7f91b006b110 con 0x7f91b010d8b0 2024-03-21T12:52:31.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.434+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91a0005240 con 0x7f91b010d8b0 2024-03-21T12:52:31.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.435+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2298626003 0 0) 0x7f91a00035b0 con 0x7f91b0109d10 2024-03-21T12:52:31.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.435+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f91b006b110 con 0x7f91b0109d10 2024-03-21T12:52:31.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.435+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f919c0018b0 con 0x7f91b0109d10 2024-03-21T12:52:31.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.435+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4097631359 0 0) 0x7f91a0005240 con 0x7f91b010d8b0 2024-03-21T12:52:31.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.435+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f91a00035b0 con 0x7f91b010d8b0 2024-03-21T12:52:31.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.435+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1489784474 0 0) 0x7f91b006b2e0 con 0x7f91b0101080 2024-03-21T12:52:31.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.435+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f91a0005240 con 0x7f91b0101080 2024-03-21T12:52:31.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.435+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f91a40044f0 con 0x7f91b0101080 2024-03-21T12:52:31.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.435+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f91a8001bf0 con 0x7f91b010d8b0 2024-03-21T12:52:31.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.435+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 858605625 0 0) 0x7f91b006b110 con 0x7f91b0109d10 2024-03-21T12:52:31.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.435+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 >> v1:172.21.15.43:6790/0 conn(0x7f91b0101080 legacy=0x7f91b01055c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:31.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.435+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 >> v1:172.21.15.43:6789/0 conn(0x7f91b010d8b0 legacy=0x7f91b01a5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:31.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.435+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b01aa740 con 0x7f91b0109d10 2024-03-21T12:52:31.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.435+0000 7f91b6eea700 1 -- 172.21.15.43:0/583242852 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b01a9730 con 0x7f91b0109d10 2024-03-21T12:52:31.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.435+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f919c003b30 con 0x7f91b0109d10 2024-03-21T12:52:31.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.435+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f919c004980 con 0x7f91b0109d10 2024-03-21T12:52:31.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.435+0000 7f91b6eea700 1 -- 172.21.15.43:0/583242852 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b01a9bf0 con 0x7f91b0109d10 2024-03-21T12:52:31.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.436+0000 7f91b6eea700 1 -- 172.21.15.43:0/583242852 --> v1:172.21.15.155:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f91b0102560 con 0x7f91b0109d10 2024-03-21T12:52:31.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.437+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7f919c0036a0 con 0x7f91b0109d10 2024-03-21T12:52:31.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.437+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 915400759 0 0) 0x7f919c088510 con 0x7f91b0109d10 2024-03-21T12:52:31.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.437+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 --> v1:172.21.15.155:6808/2978803316 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f91a007da00 con 0x7f91a0079cf0 2024-03-21T12:52:31.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.437+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (unknown 3489808268 0 0) 0x7f919c088920 con 0x7f91b0109d10 2024-03-21T12:52:31.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.439+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 <== osd.6 v1:172.21.15.155:6808/2978803316 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f91a007da00 con 0x7f91a0079cf0 2024-03-21T12:52:31.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.461+0000 7f91b6eea700 1 -- 172.21.15.43:0/583242852 --> v1:172.21.15.155:6808/2978803316 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f91b00fdf70 con 0x7f91a0079cf0 2024-03-21T12:52:31.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.462+0000 7f91adffb700 1 -- 172.21.15.43:0/583242852 <== osd.6 v1:172.21.15.155:6808/2978803316 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 992237111) 0x7f91b00fdf70 con 0x7f91a0079cf0 2024-03-21T12:52:31.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.462+0000 7f91b6eea700 1 -- 172.21.15.43:0/583242852 >> v1:172.21.15.155:6808/2978803316 conn(0x7f91a0079cf0 legacy=0x7f91a007c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:31.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.462+0000 7f91b6eea700 1 -- 172.21.15.43:0/583242852 >> v1:172.21.15.43:6800/4199387216 conn(0x7f91a006f480 legacy=0x7f91a0071940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:31.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.462+0000 7f91b6eea700 1 -- 172.21.15.43:0/583242852 >> v1:172.21.15.155:6789/0 conn(0x7f91b0109d10 legacy=0x7f91b0105cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:31.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.462+0000 7f91b5487700 1 -- 172.21.15.43:0/583242852 reap_dead start 2024-03-21T12:52:31.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.462+0000 7f91b6eea700 1 -- 172.21.15.43:0/583242852 shutdown_connections 2024-03-21T12:52:31.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.462+0000 7f91b6eea700 1 -- 172.21.15.43:0/583242852 >> 172.21.15.43:0/583242852 conn(0x7f91b00fc930 msgr2=0x7f91b0110db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:31.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.463+0000 7f91b6eea700 1 -- 172.21.15.43:0/583242852 shutdown_connections 2024-03-21T12:52:31.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:31.463+0000 7f91b6eea700 1 -- 172.21.15.43:0/583242852 wait complete. 2024-03-21T12:52:31.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.001+0000 7f3df6033700 1 -- 172.21.15.43:0/1741441276 >> v1:172.21.15.43:6789/0 conn(0x7f3df010d050 legacy=0x7f3df010f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:31.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.001+0000 7f3df6033700 1 -- 172.21.15.43:0/1741441276 shutdown_connections 2024-03-21T12:52:31.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.001+0000 7f3df6033700 1 -- 172.21.15.43:0/1741441276 >> 172.21.15.43:0/1741441276 conn(0x7f3df00772c0 msgr2=0x7f3df00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:31.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.002+0000 7f3df6033700 1 -- 172.21.15.43:0/1741441276 shutdown_connections 2024-03-21T12:52:31.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.002+0000 7f3df6033700 1 -- 172.21.15.43:0/1741441276 wait complete. 2024-03-21T12:52:31.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.002+0000 7f3df6033700 1 Processor -- start 2024-03-21T12:52:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.002+0000 7f3df6033700 1 -- start start 2024-03-21T12:52:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.002+0000 7f3df6033700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3df01a4ce0 con 0x7f3df0109420 2024-03-21T12:52:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.003+0000 7f3df6033700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3df01a5f20 con 0x7f3df010d050 2024-03-21T12:52:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.003+0000 7f3df6033700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3df01a7160 con 0x7f3df0106670 2024-03-21T12:52:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.003+0000 7f3def7fe700 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7f3df0106670 0x7f3df0102160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:52096/0 (socket says 172.21.15.43:52096) 2024-03-21T12:52:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.003+0000 7f3deeffd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3df0109420 0x7f3df0102870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54216/0 (socket says 172.21.15.43:54216) 2024-03-21T12:52:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.003+0000 7f3def7fe700 1 -- 172.21.15.43:0/724928125 learned_addr learned my addr 172.21.15.43:0/724928125 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:52:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.003+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1212564188 0 0) 0x7f3df01a7160 con 0x7f3df0106670 2024-03-21T12:52:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.003+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3dd80046c0 con 0x7f3df0106670 2024-03-21T12:52:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.003+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3083994511 0 0) 0x7f3df01a4ce0 con 0x7f3df0109420 2024-03-21T12:52:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.003+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3df01a7160 con 0x7f3df0109420 2024-03-21T12:52:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.003+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 560588591 0 0) 0x7f3df01a5f20 con 0x7f3df010d050 2024-03-21T12:52:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.003+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3df01a4ce0 con 0x7f3df010d050 2024-03-21T12:52:31.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.003+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 260786899 0 0) 0x7f3dd80046c0 con 0x7f3df0106670 2024-03-21T12:52:31.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.003+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3df01a5f20 con 0x7f3df0106670 2024-03-21T12:52:31.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.003+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f3de00019b0 con 0x7f3df0106670 2024-03-21T12:52:31.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.003+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 961478653 0 0) 0x7f3df01a7160 con 0x7f3df0109420 2024-03-21T12:52:31.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.003+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3dd80046c0 con 0x7f3df0109420 2024-03-21T12:52:31.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.003+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f3de4001830 con 0x7f3df0109420 2024-03-21T12:52:31.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.003+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 18497619 0 0) 0x7f3df01a4ce0 con 0x7f3df010d050 2024-03-21T12:52:31.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.003+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3df01a7160 con 0x7f3df010d050 2024-03-21T12:52:31.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.003+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f3de8004780 con 0x7f3df010d050 2024-03-21T12:52:31.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.003+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 <== mon.2 v1:172.21.15.43:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4240463819 0 0) 0x7f3df01a5f20 con 0x7f3df0106670 2024-03-21T12:52:31.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.003+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 >> v1:172.21.15.155:6789/0 conn(0x7f3df010d050 legacy=0x7f3df01a3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:31.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.003+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 >> v1:172.21.15.43:6789/0 conn(0x7f3df0109420 legacy=0x7f3df0102870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:31.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.003+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 --> v1:172.21.15.43:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df01a83a0 con 0x7f3df0106670 2024-03-21T12:52:31.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.003+0000 7f3df6033700 1 -- 172.21.15.43:0/724928125 --> v1:172.21.15.43:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df01a4f10 con 0x7f3df0106670 2024-03-21T12:52:31.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.004+0000 7f3df6033700 1 -- 172.21.15.43:0/724928125 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df01a5450 con 0x7f3df0106670 2024-03-21T12:52:31.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.004+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 <== mon.2 v1:172.21.15.43:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f3de0003b10 con 0x7f3df0106670 2024-03-21T12:52:31.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.004+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 <== mon.2 v1:172.21.15.43:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f3de0005c70 con 0x7f3df0106670 2024-03-21T12:52:31.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.004+0000 7f3df6033700 1 -- 172.21.15.43:0/724928125 --> v1:172.21.15.43:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3ddc000f80 con 0x7f3df0106670 2024-03-21T12:52:31.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.005+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 <== mon.2 v1:172.21.15.43:6790/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7f3de001c470 con 0x7f3df0106670 2024-03-21T12:52:31.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.005+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 <== mon.2 v1:172.21.15.43:6790/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 915400759 0 0) 0x7f3de0089640 con 0x7f3df0106670 2024-03-21T12:52:31.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.006+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 --> v1:172.21.15.43:6809/1358463812 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3dd807f770 con 0x7f3dd807ba60 2024-03-21T12:52:31.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.006+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 <== mon.2 v1:172.21.15.43:6790/0 9 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (unknown 3489808268 0 0) 0x7f3de0089ab0 con 0x7f3df0106670 2024-03-21T12:52:31.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.007+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 <== osd.2 v1:172.21.15.43:6809/1358463812 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (unknown 0 0 1302949928) 0x7f3dd807f770 con 0x7f3dd807ba60 2024-03-21T12:52:31.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.023+0000 7f3df6033700 1 -- 172.21.15.43:0/724928125 --> v1:172.21.15.43:6809/1358463812 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3ddc002ce0 con 0x7f3dd807ba60 2024-03-21T12:52:31.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.023+0000 7f3decff9700 1 -- 172.21.15.43:0/724928125 <== osd.2 v1:172.21.15.43:6809/1358463812 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 3472381573) 0x7f3ddc002ce0 con 0x7f3dd807ba60 2024-03-21T12:52:31.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.024+0000 7f3df6033700 1 -- 172.21.15.43:0/724928125 >> v1:172.21.15.43:6809/1358463812 conn(0x7f3dd807ba60 legacy=0x7f3dd807df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:31.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.024+0000 7f3df6033700 1 -- 172.21.15.43:0/724928125 >> v1:172.21.15.43:6800/4199387216 conn(0x7f3dd8071380 legacy=0x7f3dd8073840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:31.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.024+0000 7f3df6033700 1 -- 172.21.15.43:0/724928125 >> v1:172.21.15.43:6790/0 conn(0x7f3df0106670 legacy=0x7f3df0102160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:31.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.024+0000 7f3deffff700 1 -- 172.21.15.43:0/724928125 reap_dead start 2024-03-21T12:52:31.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.024+0000 7f3df6033700 1 -- 172.21.15.43:0/724928125 shutdown_connections 2024-03-21T12:52:31.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.024+0000 7f3df6033700 1 -- 172.21.15.43:0/724928125 >> 172.21.15.43:0/724928125 conn(0x7f3df00772c0 msgr2=0x7f3df010fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:31.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.024+0000 7f3df6033700 1 -- 172.21.15.43:0/724928125 shutdown_connections 2024-03-21T12:52:31.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:30.024+0000 7f3df6033700 1 -- 172.21.15.43:0/724928125 wait complete. 2024-03-21T12:52:31.755 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:52:32.120 INFO:teuthology.orchestra.run.smithi043.stdout:77309411369 2024-03-21T12:52:32.121 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-21T12:52:32.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:32 smithi155 ceph-mon[33846]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T12:52:32.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:32 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:52:32.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:32 smithi043 ceph-mon[30062]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T12:52:32.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:32 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:52:32.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:32 smithi043 ceph-mon[41699]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T12:52:32.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:32 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:52:32.940 INFO:teuthology.orchestra.run.smithi043.stdout:201863462931 2024-03-21T12:52:32.941 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-21T12:52:33.848 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:52:33.849 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:52:33.849 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:52:33.886 INFO:teuthology.orchestra.run.smithi043.stdout:236223201294 2024-03-21T12:52:33.887 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-21T12:52:34.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:34 smithi155 ceph-mon[33846]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T12:52:34.720 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:34 smithi043 ceph-mon[30062]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T12:52:34.720 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:34 smithi043 ceph-mon[41699]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T12:52:36.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:36 smithi155 ceph-mon[33846]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T12:52:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:36 smithi043 ceph-mon[30062]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T12:52:36.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:36 smithi043 ceph-mon[41699]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T12:52:36.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.987+0000 7f7c04a9d700 1 -- 172.21.15.43:0/2025535731 >> v1:172.21.15.43:6790/0 conn(0x7f7c00109d20 legacy=0x7f7c0010c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:36.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.987+0000 7f7c04a9d700 1 -- 172.21.15.43:0/2025535731 shutdown_connections 2024-03-21T12:52:36.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.987+0000 7f7c04a9d700 1 -- 172.21.15.43:0/2025535731 >> 172.21.15.43:0/2025535731 conn(0x7f7c000fc920 msgr2=0x7f7c000fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:36.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.987+0000 7f7c04a9d700 1 -- 172.21.15.43:0/2025535731 shutdown_connections 2024-03-21T12:52:36.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.987+0000 7f7c04a9d700 1 -- 172.21.15.43:0/2025535731 wait complete. 2024-03-21T12:52:36.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.988+0000 7f7c04a9d700 1 Processor -- start 2024-03-21T12:52:36.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.988+0000 7f7c04a9d700 1 -- start start 2024-03-21T12:52:36.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.988+0000 7f7c04a9d700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7c001a6e90 con 0x7f7c00109d20 2024-03-21T12:52:36.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.988+0000 7f7c04a9d700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7c001a80b0 con 0x7f7c0010d8c0 2024-03-21T12:52:36.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.988+0000 7f7c04a9d700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7c001a92f0 con 0x7f7c00101090 2024-03-21T12:52:36.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.988+0000 7f7bfe59c700 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7f7c00101090 0x7f7c00106250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:45184/0 (socket says 172.21.15.43:45184) 2024-03-21T12:52:36.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.988+0000 7f7bfe59c700 1 -- 172.21.15.43:0/2948476125 learned_addr learned my addr 172.21.15.43:0/2948476125 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:52:36.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.989+0000 7f7bf77fe700 1 -- 172.21.15.43:0/2948476125 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 269713491 0 0) 0x7f7c001a92f0 con 0x7f7c00101090 2024-03-21T12:52:36.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.989+0000 7f7bf77fe700 1 -- 172.21.15.43:0/2948476125 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bec0035b0 con 0x7f7c00101090 2024-03-21T12:52:36.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.989+0000 7f7bf77fe700 1 -- 172.21.15.43:0/2948476125 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2522638195 0 0) 0x7f7c001a6e90 con 0x7f7c00109d20 2024-03-21T12:52:36.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.989+0000 7f7bf77fe700 1 -- 172.21.15.43:0/2948476125 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c001a92f0 con 0x7f7c00109d20 2024-03-21T12:52:36.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.989+0000 7f7bf77fe700 1 -- 172.21.15.43:0/2948476125 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2047364655 0 0) 0x7f7bec0035b0 con 0x7f7c00101090 2024-03-21T12:52:36.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.989+0000 7f7bf77fe700 1 -- 172.21.15.43:0/2948476125 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7c001a6e90 con 0x7f7c00101090 2024-03-21T12:52:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.989+0000 7f7bf77fe700 1 -- 172.21.15.43:0/2948476125 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f7bf80019c0 con 0x7f7c00101090 2024-03-21T12:52:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.989+0000 7f7bf77fe700 1 -- 172.21.15.43:0/2948476125 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 423666530 0 0) 0x7f7c001a92f0 con 0x7f7c00109d20 2024-03-21T12:52:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.989+0000 7f7bf77fe700 1 -- 172.21.15.43:0/2948476125 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7bec0035b0 con 0x7f7c00109d20 2024-03-21T12:52:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.989+0000 7f7bf77fe700 1 -- 172.21.15.43:0/2948476125 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3066805213 0 0) 0x7f7c001a80b0 con 0x7f7c0010d8c0 2024-03-21T12:52:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.989+0000 7f7bf77fe700 1 -- 172.21.15.43:0/2948476125 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c001a92f0 con 0x7f7c0010d8c0 2024-03-21T12:52:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.989+0000 7f7bf77fe700 1 -- 172.21.15.43:0/2948476125 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f7be8004430 con 0x7f7c00109d20 2024-03-21T12:52:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.989+0000 7f7bf77fe700 1 -- 172.21.15.43:0/2948476125 <== mon.2 v1:172.21.15.43:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3724307561 0 0) 0x7f7c001a6e90 con 0x7f7c00101090 2024-03-21T12:52:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.989+0000 7f7bf77fe700 1 -- 172.21.15.43:0/2948476125 >> v1:172.21.15.155:6789/0 conn(0x7f7c0010d8c0 legacy=0x7f7c001a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.989+0000 7f7bf77fe700 1 -- 172.21.15.43:0/2948476125 >> v1:172.21.15.43:6789/0 conn(0x7f7c00109d20 legacy=0x7f7c001a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.989+0000 7f7bf77fe700 1 -- 172.21.15.43:0/2948476125 --> v1:172.21.15.43:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c001aa530 con 0x7f7c00101090 2024-03-21T12:52:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.989+0000 7f7c04a9d700 1 -- 172.21.15.43:0/2948476125 --> v1:172.21.15.43:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c001a9520 con 0x7f7c00101090 2024-03-21T12:52:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.989+0000 7f7c04a9d700 1 -- 172.21.15.43:0/2948476125 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c001a99e0 con 0x7f7c00101090 2024-03-21T12:52:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.989+0000 7f7bf77fe700 1 -- 172.21.15.43:0/2948476125 <== mon.2 v1:172.21.15.43:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f7bf8001d30 con 0x7f7c00101090 2024-03-21T12:52:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.989+0000 7f7bf77fe700 1 -- 172.21.15.43:0/2948476125 <== mon.2 v1:172.21.15.43:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f7bf8005e10 con 0x7f7c00101090 2024-03-21T12:52:36.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.990+0000 7f7c04a9d700 1 -- 172.21.15.43:0/2948476125 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be0005120 con 0x7f7c00101090 2024-03-21T12:52:36.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.991+0000 7f7bf77fe700 1 -- 172.21.15.43:0/2948476125 <== mon.2 v1:172.21.15.43:6790/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7f7bf8003950 con 0x7f7c00101090 2024-03-21T12:52:36.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.991+0000 7f7bf77fe700 1 -- 172.21.15.43:0/2948476125 <== mon.2 v1:172.21.15.43:6790/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 915400759 0 0) 0x7f7bf8089a40 con 0x7f7c00101090 2024-03-21T12:52:36.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:36.996+0000 7f7bf77fe700 1 -- 172.21.15.43:0/2948476125 <== mon.2 v1:172.21.15.43:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f7bf8057050 con 0x7f7c00101090 2024-03-21T12:52:37.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:37.205+0000 7f7c04a9d700 1 -- 172.21.15.43:0/2948476125 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f7be0005410 con 0x7f7c00101090 2024-03-21T12:52:37.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:37.205+0000 7f7bf77fe700 1 -- 172.21.15.43:0/2948476125 <== mon.2 v1:172.21.15.43:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (unknown 1823589948 0 1544327744) 0x7f7bf805aaa0 con 0x7f7c00101090 2024-03-21T12:52:37.205 INFO:teuthology.orchestra.run.smithi043.stdout:133143986208 2024-03-21T12:52:37.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:37.207+0000 7f7c04a9d700 1 -- 172.21.15.43:0/2948476125 >> v1:172.21.15.43:6800/4199387216 conn(0x7f7bec06f160 legacy=0x7f7bec071620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:37.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:37.207+0000 7f7c04a9d700 1 -- 172.21.15.43:0/2948476125 >> v1:172.21.15.43:6790/0 conn(0x7f7c00101090 legacy=0x7f7c00106250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:37.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:37.207+0000 7f7c04a9d700 1 -- 172.21.15.43:0/2948476125 shutdown_connections 2024-03-21T12:52:37.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:37.208+0000 7f7c04a9d700 1 -- 172.21.15.43:0/2948476125 >> 172.21.15.43:0/2948476125 conn(0x7f7c000fc920 msgr2=0x7f7c0006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:37.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:37.208+0000 7f7c04a9d700 1 -- 172.21.15.43:0/2948476125 shutdown_connections 2024-03-21T12:52:37.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:37.208+0000 7f7c04a9d700 1 -- 172.21.15.43:0/2948476125 wait complete. 2024-03-21T12:52:37.675 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:37 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2948476125' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-21T12:52:37.676 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:37 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2948476125' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-21T12:52:37.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:37 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2948476125' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-21T12:52:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.255+0000 7fa1f9256700 1 -- 172.21.15.43:0/1658957484 >> v1:172.21.15.43:6790/0 conn(0x7fa1f410ae40 legacy=0x7fa1f410d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.255+0000 7fa1f9256700 1 -- 172.21.15.43:0/1658957484 shutdown_connections 2024-03-21T12:52:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.255+0000 7fa1f9256700 1 -- 172.21.15.43:0/1658957484 >> 172.21.15.43:0/1658957484 conn(0x7fa1f40fda80 msgr2=0x7fa1f40ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.255+0000 7fa1f9256700 1 -- 172.21.15.43:0/1658957484 shutdown_connections 2024-03-21T12:52:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.255+0000 7fa1f9256700 1 -- 172.21.15.43:0/1658957484 wait complete. 2024-03-21T12:52:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.256+0000 7fa1f9256700 1 Processor -- start 2024-03-21T12:52:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.257+0000 7fa1f9256700 1 -- start start 2024-03-21T12:52:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.257+0000 7fa1f9256700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa1f41a7000 con 0x7fa1f410e9e0 2024-03-21T12:52:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.257+0000 7fa1f9256700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa1f41a8220 con 0x7fa1f410ae40 2024-03-21T12:52:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.257+0000 7fa1f9256700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa1f41a9460 con 0x7fa1f41021b0 2024-03-21T12:52:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.257+0000 7fa1f2d9d700 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7fa1f41021b0 0x7fa1f41074a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:45194/0 (socket says 172.21.15.43:45194) 2024-03-21T12:52:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.257+0000 7fa1f2d9d700 1 -- 172.21.15.43:0/1346917141 learned_addr learned my addr 172.21.15.43:0/1346917141 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:52:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.257+0000 7fa1dbfff700 1 -- 172.21.15.43:0/1346917141 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1823017192 0 0) 0x7fa1f41a9460 con 0x7fa1f41021b0 2024-03-21T12:52:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.257+0000 7fa1dbfff700 1 -- 172.21.15.43:0/1346917141 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1dc0031f0 con 0x7fa1f41021b0 2024-03-21T12:52:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.257+0000 7fa1dbfff700 1 -- 172.21.15.43:0/1346917141 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963426144 0 0) 0x7fa1f41a7000 con 0x7fa1f410e9e0 2024-03-21T12:52:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.257+0000 7fa1dbfff700 1 -- 172.21.15.43:0/1346917141 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1f41a9460 con 0x7fa1f410e9e0 2024-03-21T12:52:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.258+0000 7fa1dbfff700 1 -- 172.21.15.43:0/1346917141 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3316385616 0 0) 0x7fa1dc0031f0 con 0x7fa1f41021b0 2024-03-21T12:52:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.258+0000 7fa1dbfff700 1 -- 172.21.15.43:0/1346917141 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa1f41a7000 con 0x7fa1f41021b0 2024-03-21T12:52:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.258+0000 7fa1dbfff700 1 -- 172.21.15.43:0/1346917141 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fa1e8001930 con 0x7fa1f41021b0 2024-03-21T12:52:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.258+0000 7fa1dbfff700 1 -- 172.21.15.43:0/1346917141 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2600640987 0 0) 0x7fa1f41a9460 con 0x7fa1f410e9e0 2024-03-21T12:52:38.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.258+0000 7fa1dbfff700 1 -- 172.21.15.43:0/1346917141 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa1dc0031f0 con 0x7fa1f410e9e0 2024-03-21T12:52:38.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.258+0000 7fa1dbfff700 1 -- 172.21.15.43:0/1346917141 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fa1ec001b40 con 0x7fa1f410e9e0 2024-03-21T12:52:38.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.258+0000 7fa1dbfff700 1 -- 172.21.15.43:0/1346917141 <== mon.2 v1:172.21.15.43:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1784857885 0 0) 0x7fa1f41a7000 con 0x7fa1f41021b0 2024-03-21T12:52:38.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.258+0000 7fa1dbfff700 1 -- 172.21.15.43:0/1346917141 >> v1:172.21.15.155:6789/0 conn(0x7fa1f410ae40 legacy=0x7fa1f41a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:38.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.258+0000 7fa1dbfff700 1 -- 172.21.15.43:0/1346917141 >> v1:172.21.15.43:6789/0 conn(0x7fa1f410e9e0 legacy=0x7fa1f41a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:38.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.258+0000 7fa1dbfff700 1 -- 172.21.15.43:0/1346917141 --> v1:172.21.15.43:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f41aa6a0 con 0x7fa1f41021b0 2024-03-21T12:52:38.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.258+0000 7fa1f9256700 1 -- 172.21.15.43:0/1346917141 --> v1:172.21.15.43:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f41a7230 con 0x7fa1f41021b0 2024-03-21T12:52:38.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.258+0000 7fa1f9256700 1 -- 172.21.15.43:0/1346917141 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f41a7730 con 0x7fa1f41021b0 2024-03-21T12:52:38.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.258+0000 7fa1dbfff700 1 -- 172.21.15.43:0/1346917141 <== mon.2 v1:172.21.15.43:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7fa1e8001cc0 con 0x7fa1f41021b0 2024-03-21T12:52:38.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.258+0000 7fa1dbfff700 1 -- 172.21.15.43:0/1346917141 <== mon.2 v1:172.21.15.43:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fa1e8005d00 con 0x7fa1f41021b0 2024-03-21T12:52:38.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.259+0000 7fa1f9256700 1 -- 172.21.15.43:0/1346917141 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e0005120 con 0x7fa1f41021b0 2024-03-21T12:52:38.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.259+0000 7fa1dbfff700 1 -- 172.21.15.43:0/1346917141 <== mon.2 v1:172.21.15.43:6790/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7fa1e8003950 con 0x7fa1f41021b0 2024-03-21T12:52:38.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.260+0000 7fa1dbfff700 1 -- 172.21.15.43:0/1346917141 <== mon.2 v1:172.21.15.43:6790/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 915400759 0 0) 0x7fa1e80898c0 con 0x7fa1f41021b0 2024-03-21T12:52:38.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.264+0000 7fa1dbfff700 1 -- 172.21.15.43:0/1346917141 <== mon.2 v1:172.21.15.43:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7fa1e8056fd0 con 0x7fa1f41021b0 2024-03-21T12:52:38.396 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:52:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:52:38.435 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:52:38 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:52:38.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.492+0000 7fa1f9256700 1 -- 172.21.15.43:0/1346917141 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fa1e0005410 con 0x7fa1f41021b0 2024-03-21T12:52:38.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.492+0000 7fa1dbfff700 1 -- 172.21.15.43:0/1346917141 <== mon.2 v1:172.21.15.43:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (unknown 832126871 0 2037106864) 0x7fa1e805aa20 con 0x7fa1f41021b0 2024-03-21T12:52:38.492 INFO:teuthology.orchestra.run.smithi043.stdout:55834574894 2024-03-21T12:52:38.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.494+0000 7fa1f9256700 1 -- 172.21.15.43:0/1346917141 >> v1:172.21.15.43:6800/4199387216 conn(0x7fa1dc06eff0 legacy=0x7fa1dc0714b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:38.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.494+0000 7fa1f9256700 1 -- 172.21.15.43:0/1346917141 >> v1:172.21.15.43:6790/0 conn(0x7fa1f41021b0 legacy=0x7fa1f41074a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:38.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.494+0000 7fa1f9256700 1 -- 172.21.15.43:0/1346917141 shutdown_connections 2024-03-21T12:52:38.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.495+0000 7fa1f9256700 1 -- 172.21.15.43:0/1346917141 >> 172.21.15.43:0/1346917141 conn(0x7fa1f40fda80 msgr2=0x7fa1f4111b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:38.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.495+0000 7fa1f9256700 1 -- 172.21.15.43:0/1346917141 shutdown_connections 2024-03-21T12:52:38.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:38.495+0000 7fa1f9256700 1 -- 172.21.15.43:0/1346917141 wait complete. 2024-03-21T12:52:38.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:38 smithi155 ceph-mon[33846]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:52:38.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:38 smithi043 ceph-mon[30062]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:52:38.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:38 smithi043 ceph-mon[41699]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:52:39.690 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:39 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:52:39.690 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:39 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1346917141' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-21T12:52:39.691 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:39 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:52:39.691 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:39 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1346917141' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-21T12:52:39.692 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574892 got 55834574894 for osd.1 2024-03-21T12:52:39.692 DEBUG:teuthology.parallel:result is None 2024-03-21T12:52:39.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:39 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:52:39.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:39 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1346917141' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-21T12:52:40.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:40 smithi155 ceph-mon[33846]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T12:52:40.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:40 smithi043 ceph-mon[30062]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T12:52:40.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:40 smithi043 ceph-mon[41699]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T12:52:41.101 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986206 got 133143986208 for osd.4 2024-03-21T12:52:41.101 DEBUG:teuthology.parallel:result is None 2024-03-21T12:52:41.301 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:52:41.302 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:52:41.302 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:52:41.302 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:52:41.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.328+0000 7f1d59bbc700 1 -- 172.21.15.43:0/836089773 >> v1:172.21.15.155:6789/0 conn(0x7f1d54110bd0 legacy=0x7f1d54113070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:41.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.328+0000 7f1d59bbc700 1 -- 172.21.15.43:0/836089773 shutdown_connections 2024-03-21T12:52:41.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.328+0000 7f1d59bbc700 1 -- 172.21.15.43:0/836089773 >> 172.21.15.43:0/836089773 conn(0x7f1d540ffc10 msgr2=0x7f1d54102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:41.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.329+0000 7f1d59bbc700 1 -- 172.21.15.43:0/836089773 shutdown_connections 2024-03-21T12:52:41.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.329+0000 7f1d59bbc700 1 -- 172.21.15.43:0/836089773 wait complete. 2024-03-21T12:52:41.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.330+0000 7f1d59bbc700 1 Processor -- start 2024-03-21T12:52:41.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.330+0000 7f1d59bbc700 1 -- start start 2024-03-21T12:52:41.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.330+0000 7f1d52ffd700 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7f1d5410d030 0x7f1d54079600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:45218/0 (socket says 172.21.15.43:45218) 2024-03-21T12:52:41.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.330+0000 7f1d59bbc700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1d54109cb0 con 0x7f1d541043a0 2024-03-21T12:52:41.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.331+0000 7f1d52ffd700 1 -- 172.21.15.43:0/3572990997 learned_addr learned my addr 172.21.15.43:0/3572990997 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:52:41.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.331+0000 7f1d59bbc700 1 -- 172.21.15.43:0/3572990997 --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1d541ae860 con 0x7f1d54110bd0 2024-03-21T12:52:41.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.331+0000 7f1d59bbc700 1 -- 172.21.15.43:0/3572990997 --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1d541aea30 con 0x7f1d5410d030 2024-03-21T12:52:41.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.331+0000 7f1d50ff9700 1 -- 172.21.15.43:0/3572990997 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3491558347 0 0) 0x7f1d541ae860 con 0x7f1d54110bd0 2024-03-21T12:52:41.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.331+0000 7f1d50ff9700 1 -- 172.21.15.43:0/3572990997 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d3c0031f0 con 0x7f1d54110bd0 2024-03-21T12:52:41.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.332+0000 7f1d50ff9700 1 -- 172.21.15.43:0/3572990997 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 864378683 0 0) 0x7f1d541aea30 con 0x7f1d5410d030 2024-03-21T12:52:41.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.332+0000 7f1d50ff9700 1 -- 172.21.15.43:0/3572990997 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d541ae860 con 0x7f1d5410d030 2024-03-21T12:52:41.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.332+0000 7f1d50ff9700 1 -- 172.21.15.43:0/3572990997 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2089384433 0 0) 0x7f1d3c0031f0 con 0x7f1d54110bd0 2024-03-21T12:52:41.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.332+0000 7f1d50ff9700 1 -- 172.21.15.43:0/3572990997 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1d541aea30 con 0x7f1d54110bd0 2024-03-21T12:52:41.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.332+0000 7f1d50ff9700 1 -- 172.21.15.43:0/3572990997 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f1d4c001d30 con 0x7f1d54110bd0 2024-03-21T12:52:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.332+0000 7f1d50ff9700 1 -- 172.21.15.43:0/3572990997 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2240501330 0 0) 0x7f1d3c0035b0 con 0x7f1d541043a0 2024-03-21T12:52:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.332+0000 7f1d50ff9700 1 -- 172.21.15.43:0/3572990997 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d3c0031f0 con 0x7f1d541043a0 2024-03-21T12:52:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.332+0000 7f1d50ff9700 1 -- 172.21.15.43:0/3572990997 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 544639444 0 0) 0x7f1d541ae860 con 0x7f1d5410d030 2024-03-21T12:52:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.332+0000 7f1d50ff9700 1 -- 172.21.15.43:0/3572990997 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1d3c0035b0 con 0x7f1d5410d030 2024-03-21T12:52:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.332+0000 7f1d50ff9700 1 -- 172.21.15.43:0/3572990997 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f1d480018b0 con 0x7f1d5410d030 2024-03-21T12:52:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.332+0000 7f1d50ff9700 1 -- 172.21.15.43:0/3572990997 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 132146997 0 0) 0x7f1d541aea30 con 0x7f1d54110bd0 2024-03-21T12:52:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.332+0000 7f1d50ff9700 1 -- 172.21.15.43:0/3572990997 >> v1:172.21.15.43:6790/0 conn(0x7f1d5410d030 legacy=0x7f1d54079600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.332+0000 7f1d50ff9700 1 -- 172.21.15.43:0/3572990997 >> v1:172.21.15.43:6789/0 conn(0x7f1d541043a0 legacy=0x7f1d54109450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.332+0000 7f1d50ff9700 1 -- 172.21.15.43:0/3572990997 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d541aec40 con 0x7f1d54110bd0 2024-03-21T12:52:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.332+0000 7f1d59bbc700 1 -- 172.21.15.43:0/3572990997 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d541aef30 con 0x7f1d54110bd0 2024-03-21T12:52:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.332+0000 7f1d59bbc700 1 -- 172.21.15.43:0/3572990997 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d541af3f0 con 0x7f1d54110bd0 2024-03-21T12:52:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.333+0000 7f1d50ff9700 1 -- 172.21.15.43:0/3572990997 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f1d4c004a20 con 0x7f1d54110bd0 2024-03-21T12:52:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.333+0000 7f1d50ff9700 1 -- 172.21.15.43:0/3572990997 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f1d4c005050 con 0x7f1d54110bd0 2024-03-21T12:52:41.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.333+0000 7f1d59bbc700 1 -- 172.21.15.43:0/3572990997 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d54105880 con 0x7f1d54110bd0 2024-03-21T12:52:41.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.335+0000 7f1d50ff9700 1 -- 172.21.15.43:0/3572990997 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7f1d4c003b00 con 0x7f1d54110bd0 2024-03-21T12:52:41.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.336+0000 7f1d50ff9700 1 -- 172.21.15.43:0/3572990997 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 915400759 0 0) 0x7f1d4c0889f0 con 0x7f1d54110bd0 2024-03-21T12:52:41.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.341+0000 7f1d50ff9700 1 -- 172.21.15.43:0/3572990997 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f1d4c055fc0 con 0x7f1d54110bd0 2024-03-21T12:52:41.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.558+0000 7f1d59bbc700 1 -- 172.21.15.43:0/3572990997 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f1d54107c90 con 0x7f1d54110bd0 2024-03-21T12:52:41.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.559+0000 7f1d50ff9700 1 -- 172.21.15.43:0/3572990997 <== mon.1 v1:172.21.15.155:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (unknown 2131975755 0 616938113) 0x7f1d4c059a10 con 0x7f1d54110bd0 2024-03-21T12:52:41.559 INFO:teuthology.orchestra.run.smithi043.stdout:167503724572 2024-03-21T12:52:41.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.561+0000 7f1d59bbc700 1 -- 172.21.15.43:0/3572990997 >> v1:172.21.15.43:6800/4199387216 conn(0x7f1d3c06f320 legacy=0x7f1d3c0717e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:41.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.561+0000 7f1d59bbc700 1 -- 172.21.15.43:0/3572990997 >> v1:172.21.15.155:6789/0 conn(0x7f1d54110bd0 legacy=0x7f1d54079d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:41.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.561+0000 7f1d59bbc700 1 -- 172.21.15.43:0/3572990997 shutdown_connections 2024-03-21T12:52:41.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.561+0000 7f1d59bbc700 1 -- 172.21.15.43:0/3572990997 >> 172.21.15.43:0/3572990997 conn(0x7f1d540ffc10 msgr2=0x7f1d541140d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:41.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.561+0000 7f1d59bbc700 1 -- 172.21.15.43:0/3572990997 shutdown_connections 2024-03-21T12:52:41.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:41.561+0000 7f1d59bbc700 1 -- 172.21.15.43:0/3572990997 wait complete. 2024-03-21T12:52:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.167+0000 7f9eb5e24700 1 -- 172.21.15.43:0/3243508559 >> v1:172.21.15.43:6789/0 conn(0x7f9eb0106f50 legacy=0x7f9eb0107330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.167+0000 7f9eb5e24700 1 -- 172.21.15.43:0/3243508559 shutdown_connections 2024-03-21T12:52:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.167+0000 7f9eb5e24700 1 -- 172.21.15.43:0/3243508559 >> 172.21.15.43:0/3243508559 conn(0x7f9eb0077270 msgr2=0x7f9eb00fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.167+0000 7f9eb5e24700 1 -- 172.21.15.43:0/3243508559 shutdown_connections 2024-03-21T12:52:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.167+0000 7f9eb5e24700 1 -- 172.21.15.43:0/3243508559 wait complete. 2024-03-21T12:52:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.168+0000 7f9eb5e24700 1 Processor -- start 2024-03-21T12:52:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.168+0000 7f9eb5e24700 1 -- start start 2024-03-21T12:52:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.168+0000 7f9eb5e24700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9eb01a9260 con 0x7f9eb010add0 2024-03-21T12:52:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.168+0000 7f9eb5e24700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9eb01aa480 con 0x7f9eb0106f50 2024-03-21T12:52:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.168+0000 7f9eb5e24700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9eb01ab6a0 con 0x7f9eb010eaa0 2024-03-21T12:52:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.168+0000 7f9ea6ffd700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9eb010add0 0x7f9eb01a4340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34318/0 (socket says 172.21.15.43:34318) 2024-03-21T12:52:42.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.168+0000 7f9ea6ffd700 1 -- 172.21.15.43:0/3695175905 learned_addr learned my addr 172.21.15.43:0/3695175905 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:52:42.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.168+0000 7f9eaffff700 1 --1- 172.21.15.43:0/3695175905 >> v1:172.21.15.43:6790/0 conn(0x7f9eb010eaa0 0x7f9eb01a7a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:53636/0 (socket says 172.21.15.43:53636) 2024-03-21T12:52:42.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.169+0000 7f9ead7fa700 1 -- 172.21.15.43:0/3695175905 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3188963059 0 0) 0x7f9eb01a9260 con 0x7f9eb010add0 2024-03-21T12:52:42.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.169+0000 7f9ead7fa700 1 -- 172.21.15.43:0/3695175905 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e9c0035b0 con 0x7f9eb010add0 2024-03-21T12:52:42.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.169+0000 7f9ead7fa700 1 -- 172.21.15.43:0/3695175905 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2500737993 0 0) 0x7f9eb01ab6a0 con 0x7f9eb010eaa0 2024-03-21T12:52:42.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.169+0000 7f9ead7fa700 1 -- 172.21.15.43:0/3695175905 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9eb01a9260 con 0x7f9eb010eaa0 2024-03-21T12:52:42.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.169+0000 7f9ead7fa700 1 -- 172.21.15.43:0/3695175905 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2672194627 0 0) 0x7f9eb01aa480 con 0x7f9eb0106f50 2024-03-21T12:52:42.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.169+0000 7f9ead7fa700 1 -- 172.21.15.43:0/3695175905 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9eb01ab6a0 con 0x7f9eb0106f50 2024-03-21T12:52:42.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.169+0000 7f9ead7fa700 1 -- 172.21.15.43:0/3695175905 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 145024116 0 0) 0x7f9e9c0035b0 con 0x7f9eb010add0 2024-03-21T12:52:42.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.169+0000 7f9ead7fa700 1 -- 172.21.15.43:0/3695175905 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9eb01aa480 con 0x7f9eb010add0 2024-03-21T12:52:42.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.169+0000 7f9ead7fa700 1 -- 172.21.15.43:0/3695175905 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f9e980018f0 con 0x7f9eb010add0 2024-03-21T12:52:42.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.169+0000 7f9ead7fa700 1 -- 172.21.15.43:0/3695175905 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 696132770 0 0) 0x7f9eb01a9260 con 0x7f9eb010eaa0 2024-03-21T12:52:42.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.169+0000 7f9ead7fa700 1 -- 172.21.15.43:0/3695175905 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9e9c0035b0 con 0x7f9eb010eaa0 2024-03-21T12:52:42.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.169+0000 7f9ead7fa700 1 -- 172.21.15.43:0/3695175905 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f9ea80015b0 con 0x7f9eb010eaa0 2024-03-21T12:52:42.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.169+0000 7f9ead7fa700 1 -- 172.21.15.43:0/3695175905 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4112854629 0 0) 0x7f9eb01ab6a0 con 0x7f9eb0106f50 2024-03-21T12:52:42.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.169+0000 7f9ead7fa700 1 -- 172.21.15.43:0/3695175905 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9eb01a9260 con 0x7f9eb0106f50 2024-03-21T12:52:42.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.169+0000 7f9ead7fa700 1 -- 172.21.15.43:0/3695175905 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f9ea0004430 con 0x7f9eb0106f50 2024-03-21T12:52:42.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.169+0000 7f9ead7fa700 1 -- 172.21.15.43:0/3695175905 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3579648426 0 0) 0x7f9eb01aa480 con 0x7f9eb010add0 2024-03-21T12:52:42.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.169+0000 7f9ead7fa700 1 -- 172.21.15.43:0/3695175905 >> v1:172.21.15.43:6790/0 conn(0x7f9eb010eaa0 legacy=0x7f9eb01a7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:42.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.169+0000 7f9ead7fa700 1 -- 172.21.15.43:0/3695175905 >> v1:172.21.15.155:6789/0 conn(0x7f9eb0106f50 legacy=0x7f9eb0102e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:42.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.169+0000 7f9ead7fa700 1 -- 172.21.15.43:0/3695175905 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb01ac8e0 con 0x7f9eb010add0 2024-03-21T12:52:42.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.169+0000 7f9eb5e24700 1 -- 172.21.15.43:0/3695175905 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb01ab8d0 con 0x7f9eb010add0 2024-03-21T12:52:42.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.169+0000 7f9eb5e24700 1 -- 172.21.15.43:0/3695175905 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb01abd90 con 0x7f9eb010add0 2024-03-21T12:52:42.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.169+0000 7f9ead7fa700 1 -- 172.21.15.43:0/3695175905 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f9e98003b30 con 0x7f9eb010add0 2024-03-21T12:52:42.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.169+0000 7f9ead7fa700 1 -- 172.21.15.43:0/3695175905 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f9e98005cc0 con 0x7f9eb010add0 2024-03-21T12:52:42.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.170+0000 7f9eb5e24700 1 -- 172.21.15.43:0/3695175905 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e90005120 con 0x7f9eb010add0 2024-03-21T12:52:42.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.171+0000 7f9ead7fa700 1 -- 172.21.15.43:0/3695175905 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7f9e980036a0 con 0x7f9eb010add0 2024-03-21T12:52:42.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.171+0000 7f9ead7fa700 1 -- 172.21.15.43:0/3695175905 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 915400759 0 0) 0x7f9e98089740 con 0x7f9eb010add0 2024-03-21T12:52:42.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.176+0000 7f9ead7fa700 1 -- 172.21.15.43:0/3695175905 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f9e98005ca0 con 0x7f9eb010add0 2024-03-21T12:52:42.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.384+0000 7f9eb5e24700 1 -- 172.21.15.43:0/3695175905 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f9e90005410 con 0x7f9eb010add0 2024-03-21T12:52:42.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.384+0000 7f9ead7fa700 1 -- 172.21.15.43:0/3695175905 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (unknown 574334944 0 190233549) 0x7f9e98056d70 con 0x7f9eb010add0 2024-03-21T12:52:42.386 INFO:teuthology.orchestra.run.smithi043.stdout:34359738421 2024-03-21T12:52:42.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.386+0000 7f9eb5e24700 1 -- 172.21.15.43:0/3695175905 >> v1:172.21.15.43:6800/4199387216 conn(0x7f9e9c06f180 legacy=0x7f9e9c071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:42.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.386+0000 7f9eb5e24700 1 -- 172.21.15.43:0/3695175905 >> v1:172.21.15.43:6789/0 conn(0x7f9eb010add0 legacy=0x7f9eb01a4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:42.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.387+0000 7f9eb5e24700 1 -- 172.21.15.43:0/3695175905 shutdown_connections 2024-03-21T12:52:42.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.387+0000 7f9eb5e24700 1 -- 172.21.15.43:0/3695175905 >> 172.21.15.43:0/3695175905 conn(0x7f9eb0077270 msgr2=0x7f9eb0111d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:42.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.387+0000 7f9eb5e24700 1 -- 172.21.15.43:0/3695175905 shutdown_connections 2024-03-21T12:52:42.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:42.387+0000 7f9eb5e24700 1 -- 172.21.15.43:0/3695175905 wait complete. 2024-03-21T12:52:42.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:42 smithi155 ceph-mon[33846]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:52:42.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3572990997' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-21T12:52:42.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3695175905' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-21T12:52:42.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:42 smithi043 ceph-mon[30062]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:52:42.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3572990997' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-21T12:52:42.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3695175905' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-21T12:52:42.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:42 smithi043 ceph-mon[41699]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:52:42.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3572990997' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-21T12:52:42.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3695175905' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-21T12:52:44.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:44 smithi155 ceph-mon[33846]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:52:44.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:44 smithi043 ceph-mon[30062]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:52:44.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:44 smithi043 ceph-mon[41699]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:52:45.245 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724570 got 167503724572 for osd.5 2024-03-21T12:52:45.246 DEBUG:teuthology.parallel:result is None 2024-03-21T12:52:45.689 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738418 got 34359738421 for osd.0 2024-03-21T12:52:45.690 DEBUG:teuthology.parallel:result is None 2024-03-21T12:52:46.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:46 smithi155 ceph-mon[33846]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:52:46.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:46 smithi043 ceph-mon[30062]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:52:46.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:46 smithi043 ceph-mon[41699]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:52:47.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.179+0000 7fa0bd689700 1 -- 172.21.15.43:0/1277551646 >> v1:172.21.15.43:6790/0 conn(0x7fa0b8109d10 legacy=0x7fa0b810c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:47.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.179+0000 7fa0bd689700 1 -- 172.21.15.43:0/1277551646 shutdown_connections 2024-03-21T12:52:47.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.179+0000 7fa0bd689700 1 -- 172.21.15.43:0/1277551646 >> 172.21.15.43:0/1277551646 conn(0x7fa0b80fc930 msgr2=0x7fa0b80fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:47.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.179+0000 7fa0bd689700 1 -- 172.21.15.43:0/1277551646 shutdown_connections 2024-03-21T12:52:47.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.179+0000 7fa0bd689700 1 -- 172.21.15.43:0/1277551646 wait complete. 2024-03-21T12:52:47.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.180+0000 7fa0bd689700 1 Processor -- start 2024-03-21T12:52:47.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.180+0000 7fa0bd689700 1 -- start start 2024-03-21T12:52:47.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.180+0000 7fa0bd689700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa0b81a5ed0 con 0x7fa0b810d8b0 2024-03-21T12:52:47.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.180+0000 7fa0bd689700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa0b81a70f0 con 0x7fa0b8109d10 2024-03-21T12:52:47.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.180+0000 7fa0bd689700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa0b81a8330 con 0x7fa0b8101080 2024-03-21T12:52:47.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.180+0000 7fa0b6ffd700 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7fa0b8101080 0x7fa0b8106360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:53662/0 (socket says 172.21.15.43:53662) 2024-03-21T12:52:47.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.180+0000 7fa0b6ffd700 1 -- 172.21.15.43:0/1209018485 learned_addr learned my addr 172.21.15.43:0/1209018485 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:52:47.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.180+0000 7fa0b77fe700 1 --1- 172.21.15.43:0/1209018485 >> v1:172.21.15.43:6789/0 conn(0x7fa0b810d8b0 0x7fa0b81a4590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34332/0 (socket says 172.21.15.43:34332) 2024-03-21T12:52:47.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.181+0000 7fa0affff700 1 -- 172.21.15.43:0/1209018485 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2097240682 0 0) 0x7fa0b81a8330 con 0x7fa0b8101080 2024-03-21T12:52:47.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.181+0000 7fa0affff700 1 -- 172.21.15.43:0/1209018485 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0a40035b0 con 0x7fa0b8101080 2024-03-21T12:52:47.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.181+0000 7fa0affff700 1 -- 172.21.15.43:0/1209018485 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 516409088 0 0) 0x7fa0b81a5ed0 con 0x7fa0b810d8b0 2024-03-21T12:52:47.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.181+0000 7fa0affff700 1 -- 172.21.15.43:0/1209018485 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0b81a8330 con 0x7fa0b810d8b0 2024-03-21T12:52:47.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.181+0000 7fa0affff700 1 -- 172.21.15.43:0/1209018485 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 59048105 0 0) 0x7fa0b81a70f0 con 0x7fa0b8109d10 2024-03-21T12:52:47.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.181+0000 7fa0affff700 1 -- 172.21.15.43:0/1209018485 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0b81a5ed0 con 0x7fa0b8109d10 2024-03-21T12:52:47.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.181+0000 7fa0affff700 1 -- 172.21.15.43:0/1209018485 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3293978815 0 0) 0x7fa0a40035b0 con 0x7fa0b8101080 2024-03-21T12:52:47.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.181+0000 7fa0affff700 1 -- 172.21.15.43:0/1209018485 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa0b81a70f0 con 0x7fa0b8101080 2024-03-21T12:52:47.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.181+0000 7fa0affff700 1 -- 172.21.15.43:0/1209018485 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fa0a8001bb0 con 0x7fa0b8101080 2024-03-21T12:52:47.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.181+0000 7fa0affff700 1 -- 172.21.15.43:0/1209018485 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1313244108 0 0) 0x7fa0b81a8330 con 0x7fa0b810d8b0 2024-03-21T12:52:47.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.181+0000 7fa0affff700 1 -- 172.21.15.43:0/1209018485 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa0a40035b0 con 0x7fa0b810d8b0 2024-03-21T12:52:47.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.181+0000 7fa0affff700 1 -- 172.21.15.43:0/1209018485 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fa0b00019b0 con 0x7fa0b810d8b0 2024-03-21T12:52:47.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.181+0000 7fa0affff700 1 -- 172.21.15.43:0/1209018485 <== mon.2 v1:172.21.15.43:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1092554975 0 0) 0x7fa0b81a70f0 con 0x7fa0b8101080 2024-03-21T12:52:47.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.181+0000 7fa0affff700 1 -- 172.21.15.43:0/1209018485 >> v1:172.21.15.155:6789/0 conn(0x7fa0b8109d10 legacy=0x7fa0b81a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:47.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.181+0000 7fa0affff700 1 -- 172.21.15.43:0/1209018485 >> v1:172.21.15.43:6789/0 conn(0x7fa0b810d8b0 legacy=0x7fa0b81a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:47.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.181+0000 7fa0affff700 1 -- 172.21.15.43:0/1209018485 --> v1:172.21.15.43:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b81a9570 con 0x7fa0b8101080 2024-03-21T12:52:47.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.181+0000 7fa0bd689700 1 -- 172.21.15.43:0/1209018485 --> v1:172.21.15.43:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0b81a6100 con 0x7fa0b8101080 2024-03-21T12:52:47.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.181+0000 7fa0bd689700 1 -- 172.21.15.43:0/1209018485 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0b81a65c0 con 0x7fa0b8101080 2024-03-21T12:52:47.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.181+0000 7fa0affff700 1 -- 172.21.15.43:0/1209018485 <== mon.2 v1:172.21.15.43:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7fa0a8003eb0 con 0x7fa0b8101080 2024-03-21T12:52:47.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.181+0000 7fa0affff700 1 -- 172.21.15.43:0/1209018485 <== mon.2 v1:172.21.15.43:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fa0a8005f10 con 0x7fa0b8101080 2024-03-21T12:52:47.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.182+0000 7fa0bd689700 1 -- 172.21.15.43:0/1209018485 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa098005120 con 0x7fa0b8101080 2024-03-21T12:52:47.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.183+0000 7fa0affff700 1 -- 172.21.15.43:0/1209018485 <== mon.2 v1:172.21.15.43:6790/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7fa0a801c710 con 0x7fa0b8101080 2024-03-21T12:52:47.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.183+0000 7fa0affff700 1 -- 172.21.15.43:0/1209018485 <== mon.2 v1:172.21.15.43:6790/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 915400759 0 0) 0x7fa0a8089970 con 0x7fa0b8101080 2024-03-21T12:52:47.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.188+0000 7fa0affff700 1 -- 172.21.15.43:0/1209018485 <== mon.2 v1:172.21.15.43:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7fa0a8056fa0 con 0x7fa0b8101080 2024-03-21T12:52:47.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.404+0000 7fa0bd689700 1 -- 172.21.15.43:0/1209018485 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fa098005410 con 0x7fa0b8101080 2024-03-21T12:52:47.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.405+0000 7fa0affff700 1 -- 172.21.15.43:0/1209018485 <== mon.2 v1:172.21.15.43:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (unknown 92182286 0 3499689557) 0x7fa0a805a9f0 con 0x7fa0b8101080 2024-03-21T12:52:47.405 INFO:teuthology.orchestra.run.smithi043.stdout:77309411372 2024-03-21T12:52:47.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.407+0000 7fa0bd689700 1 -- 172.21.15.43:0/1209018485 >> v1:172.21.15.43:6800/4199387216 conn(0x7fa0a40710e0 legacy=0x7fa0a40735a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:47.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.407+0000 7fa0bd689700 1 -- 172.21.15.43:0/1209018485 >> v1:172.21.15.43:6790/0 conn(0x7fa0b8101080 legacy=0x7fa0b8106360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:47.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.407+0000 7fa0bd689700 1 -- 172.21.15.43:0/1209018485 shutdown_connections 2024-03-21T12:52:47.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.407+0000 7fa0bd689700 1 -- 172.21.15.43:0/1209018485 >> 172.21.15.43:0/1209018485 conn(0x7fa0b80fc930 msgr2=0x7fa0b8110560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:47.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.407+0000 7fa0bd689700 1 -- 172.21.15.43:0/1209018485 shutdown_connections 2024-03-21T12:52:47.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:47.407+0000 7fa0bd689700 1 -- 172.21.15.43:0/1209018485 wait complete. 2024-03-21T12:52:47.522 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:47 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:52:47.523 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1209018485' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-21T12:52:47.523 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:47 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:52:47.523 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1209018485' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-21T12:52:47.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:47 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:52:47.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1209018485' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-21T12:52:48.150 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:52:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:52:48.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:52:48 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:52:48.726 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:48 smithi043 ceph-mon[30062]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:52:48.727 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:48 smithi043 ceph-mon[41699]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:52:48.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:48 smithi155 ceph-mon[33846]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:52:49.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.347+0000 7fc460035700 1 -- 172.21.15.43:0/669987380 >> v1:172.21.15.43:6790/0 conn(0x7fc45810aef0 legacy=0x7fc45810d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:49.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.347+0000 7fc460035700 1 -- 172.21.15.43:0/669987380 shutdown_connections 2024-03-21T12:52:49.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.347+0000 7fc460035700 1 -- 172.21.15.43:0/669987380 >> 172.21.15.43:0/669987380 conn(0x7fc4580fc930 msgr2=0x7fc4580fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:49.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.347+0000 7fc460035700 1 -- 172.21.15.43:0/669987380 shutdown_connections 2024-03-21T12:52:49.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.347+0000 7fc460035700 1 -- 172.21.15.43:0/669987380 wait complete. 2024-03-21T12:52:49.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.348+0000 7fc460035700 1 Processor -- start 2024-03-21T12:52:49.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.348+0000 7fc460035700 1 -- start start 2024-03-21T12:52:49.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.348+0000 7fc460035700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc4581a6da0 con 0x7fc45810aef0 2024-03-21T12:52:49.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.348+0000 7fc460035700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc4581a7fc0 con 0x7fc45810ea00 2024-03-21T12:52:49.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.348+0000 7fc45ddd1700 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7fc4581030e0 0x7fc45810a4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:53672/0 (socket says 172.21.15.43:53672) 2024-03-21T12:52:49.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.348+0000 7fc45ddd1700 1 -- 172.21.15.43:0/1748581214 learned_addr learned my addr 172.21.15.43:0/1748581214 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:52:49.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.348+0000 7fc460035700 1 -- 172.21.15.43:0/1748581214 --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc4581a91e0 con 0x7fc4581030e0 2024-03-21T12:52:49.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.348+0000 7fc456ffd700 1 -- 172.21.15.43:0/1748581214 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2563103986 0 0) 0x7fc4581a91e0 con 0x7fc4581030e0 2024-03-21T12:52:49.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.348+0000 7fc456ffd700 1 -- 172.21.15.43:0/1748581214 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4400031f0 con 0x7fc4581030e0 2024-03-21T12:52:49.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.348+0000 7fc456ffd700 1 -- 172.21.15.43:0/1748581214 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3366531031 0 0) 0x7fc4581a6da0 con 0x7fc45810aef0 2024-03-21T12:52:49.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.348+0000 7fc456ffd700 1 -- 172.21.15.43:0/1748581214 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4581a91e0 con 0x7fc45810aef0 2024-03-21T12:52:49.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.349+0000 7fc456ffd700 1 -- 172.21.15.43:0/1748581214 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 204719306 0 0) 0x7fc4581a7fc0 con 0x7fc45810ea00 2024-03-21T12:52:49.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.349+0000 7fc456ffd700 1 -- 172.21.15.43:0/1748581214 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4581a6da0 con 0x7fc45810ea00 2024-03-21T12:52:49.083 INFO:teuthology.orchestra.run.smithi043.stdout:111669149735 2024-03-21T12:52:49.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.349+0000 7fc456ffd700 1 -- 172.21.15.43:0/1748581214 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3413808157 0 0) 0x7fc4400031f0 con 0x7fc4581030e0 2024-03-21T12:52:49.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.349+0000 7fc456ffd700 1 -- 172.21.15.43:0/1748581214 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc4581a7fc0 con 0x7fc4581030e0 2024-03-21T12:52:49.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.349+0000 7fc456ffd700 1 -- 172.21.15.43:0/1748581214 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fc450001d40 con 0x7fc4581030e0 2024-03-21T12:52:49.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.349+0000 7fc456ffd700 1 -- 172.21.15.43:0/1748581214 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 919100896 0 0) 0x7fc4581a91e0 con 0x7fc45810aef0 2024-03-21T12:52:49.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.349+0000 7fc456ffd700 1 -- 172.21.15.43:0/1748581214 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc4400031f0 con 0x7fc45810aef0 2024-03-21T12:52:49.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.349+0000 7fc456ffd700 1 -- 172.21.15.43:0/1748581214 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fc44c001710 con 0x7fc45810aef0 2024-03-21T12:52:49.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.349+0000 7fc456ffd700 1 -- 172.21.15.43:0/1748581214 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3494868302 0 0) 0x7fc4581a6da0 con 0x7fc45810ea00 2024-03-21T12:52:49.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.349+0000 7fc456ffd700 1 -- 172.21.15.43:0/1748581214 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc4581a91e0 con 0x7fc45810ea00 2024-03-21T12:52:49.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.349+0000 7fc456ffd700 1 -- 172.21.15.43:0/1748581214 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fc448001a30 con 0x7fc45810ea00 2024-03-21T12:52:49.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.349+0000 7fc456ffd700 1 -- 172.21.15.43:0/1748581214 <== mon.2 v1:172.21.15.43:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3421393657 0 0) 0x7fc4581a7fc0 con 0x7fc4581030e0 2024-03-21T12:52:49.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.349+0000 7fc456ffd700 1 -- 172.21.15.43:0/1748581214 >> v1:172.21.15.155:6789/0 conn(0x7fc45810ea00 legacy=0x7fc4581a5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:49.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.349+0000 7fc456ffd700 1 -- 172.21.15.43:0/1748581214 >> v1:172.21.15.43:6789/0 conn(0x7fc45810aef0 legacy=0x7fc4581a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:49.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.349+0000 7fc456ffd700 1 -- 172.21.15.43:0/1748581214 --> v1:172.21.15.43:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4581aa420 con 0x7fc4581030e0 2024-03-21T12:52:49.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.349+0000 7fc460035700 1 -- 172.21.15.43:0/1748581214 --> v1:172.21.15.43:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4581a6fd0 con 0x7fc4581030e0 2024-03-21T12:52:49.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.349+0000 7fc460035700 1 -- 172.21.15.43:0/1748581214 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4581a7490 con 0x7fc4581030e0 2024-03-21T12:52:49.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.349+0000 7fc456ffd700 1 -- 172.21.15.43:0/1748581214 <== mon.2 v1:172.21.15.43:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7fc450004160 con 0x7fc4581030e0 2024-03-21T12:52:49.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.349+0000 7fc456ffd700 1 -- 172.21.15.43:0/1748581214 <== mon.2 v1:172.21.15.43:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fc450006120 con 0x7fc4581030e0 2024-03-21T12:52:49.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.350+0000 7fc460035700 1 -- 172.21.15.43:0/1748581214 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc444005120 con 0x7fc4581030e0 2024-03-21T12:52:49.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.350+0000 7fc456ffd700 1 -- 172.21.15.43:0/1748581214 <== mon.2 v1:172.21.15.43:6790/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7fc450003cd0 con 0x7fc4581030e0 2024-03-21T12:52:49.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.351+0000 7fc456ffd700 1 -- 172.21.15.43:0/1748581214 <== mon.2 v1:172.21.15.43:6790/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 915400759 0 0) 0x7fc450089b70 con 0x7fc4581030e0 2024-03-21T12:52:49.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.355+0000 7fc456ffd700 1 -- 172.21.15.43:0/1748581214 <== mon.2 v1:172.21.15.43:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7fc450057140 con 0x7fc4581030e0 2024-03-21T12:52:49.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.568+0000 7fc460035700 1 -- 172.21.15.43:0/1748581214 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fc444005410 con 0x7fc4581030e0 2024-03-21T12:52:49.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.569+0000 7fc456ffd700 1 -- 172.21.15.43:0/1748581214 <== mon.2 v1:172.21.15.43:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (unknown 383520633 0 787726074) 0x7fc45005ab90 con 0x7fc4581030e0 2024-03-21T12:52:49.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.571+0000 7fc460035700 1 -- 172.21.15.43:0/1748581214 >> v1:172.21.15.43:6800/4199387216 conn(0x7fc44006f1e0 legacy=0x7fc4400716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:49.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.571+0000 7fc460035700 1 -- 172.21.15.43:0/1748581214 >> v1:172.21.15.43:6790/0 conn(0x7fc4581030e0 legacy=0x7fc45810a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:49.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.571+0000 7fc460035700 1 -- 172.21.15.43:0/1748581214 shutdown_connections 2024-03-21T12:52:49.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.571+0000 7fc460035700 1 -- 172.21.15.43:0/1748581214 >> 172.21.15.43:0/1748581214 conn(0x7fc4580fc930 msgr2=0x7fc458110e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:49.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.571+0000 7fc460035700 1 -- 172.21.15.43:0/1748581214 shutdown_connections 2024-03-21T12:52:49.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:48.571+0000 7fc460035700 1 -- 172.21.15.43:0/1748581214 wait complete. 2024-03-21T12:52:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.078+0000 7f6485f7a700 1 -- 172.21.15.43:0/1185274515 >> v1:172.21.15.43:6790/0 conn(0x7f648010ea60 legacy=0x7f6480110f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.078+0000 7f6485f7a700 1 -- 172.21.15.43:0/1185274515 shutdown_connections 2024-03-21T12:52:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.078+0000 7f6485f7a700 1 -- 172.21.15.43:0/1185274515 >> 172.21.15.43:0/1185274515 conn(0x7f6480076180 msgr2=0x7f6480076590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.078+0000 7f6485f7a700 1 -- 172.21.15.43:0/1185274515 shutdown_connections 2024-03-21T12:52:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.078+0000 7f6485f7a700 1 -- 172.21.15.43:0/1185274515 wait complete. 2024-03-21T12:52:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.078+0000 7f6485f7a700 1 Processor -- start 2024-03-21T12:52:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.078+0000 7f6485f7a700 1 -- start start 2024-03-21T12:52:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.079+0000 7f6485f7a700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f64801a9130 con 0x7f6480100960 2024-03-21T12:52:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.079+0000 7f6485f7a700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f64801aa310 con 0x7f648010aef0 2024-03-21T12:52:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.079+0000 7f6485f7a700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f64801ab4f0 con 0x7f648010ea60 2024-03-21T12:52:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.079+0000 7f647f7fe700 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6480100960 0x7f6480107160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:34372/0 (socket says 172.21.15.43:34372) 2024-03-21T12:52:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.079+0000 7f647f7fe700 1 -- 172.21.15.43:0/563724797 learned_addr learned my addr 172.21.15.43:0/563724797 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:52:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.079+0000 7f647ffff700 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7f648010ea60 0x7f64801a7a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:53688/0 (socket says 172.21.15.43:53688) 2024-03-21T12:52:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.079+0000 7f647cff9700 1 -- 172.21.15.43:0/563724797 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1171148770 0 0) 0x7f64801a9130 con 0x7f6480100960 2024-03-21T12:52:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.079+0000 7f647cff9700 1 -- 172.21.15.43:0/563724797 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64680031f0 con 0x7f6480100960 2024-03-21T12:52:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.079+0000 7f647cff9700 1 -- 172.21.15.43:0/563724797 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 42107700 0 0) 0x7f64801ab4f0 con 0x7f648010ea60 2024-03-21T12:52:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.079+0000 7f647cff9700 1 -- 172.21.15.43:0/563724797 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64801a9130 con 0x7f648010ea60 2024-03-21T12:52:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.079+0000 7f647cff9700 1 -- 172.21.15.43:0/563724797 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2524626262 0 0) 0x7f64801aa310 con 0x7f648010aef0 2024-03-21T12:52:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.079+0000 7f647cff9700 1 -- 172.21.15.43:0/563724797 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64801ab4f0 con 0x7f648010aef0 2024-03-21T12:52:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.079+0000 7f647cff9700 1 -- 172.21.15.43:0/563724797 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1374225098 0 0) 0x7f64680031f0 con 0x7f6480100960 2024-03-21T12:52:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.079+0000 7f647cff9700 1 -- 172.21.15.43:0/563724797 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f64801aa310 con 0x7f6480100960 2024-03-21T12:52:49.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.079+0000 7f647cff9700 1 -- 172.21.15.43:0/563724797 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f64700019c0 con 0x7f6480100960 2024-03-21T12:52:49.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.079+0000 7f647cff9700 1 -- 172.21.15.43:0/563724797 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2608562327 0 0) 0x7f64801a9130 con 0x7f648010ea60 2024-03-21T12:52:49.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.079+0000 7f647cff9700 1 -- 172.21.15.43:0/563724797 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f64680031f0 con 0x7f648010ea60 2024-03-21T12:52:49.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.079+0000 7f647cff9700 1 -- 172.21.15.43:0/563724797 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f64780046a0 con 0x7f648010ea60 2024-03-21T12:52:49.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.079+0000 7f647cff9700 1 -- 172.21.15.43:0/563724797 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1383729046 0 0) 0x7f64801aa310 con 0x7f6480100960 2024-03-21T12:52:49.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.080+0000 7f647cff9700 1 -- 172.21.15.43:0/563724797 >> v1:172.21.15.43:6790/0 conn(0x7f648010ea60 legacy=0x7f64801a7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:49.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.080+0000 7f647cff9700 1 -- 172.21.15.43:0/563724797 >> v1:172.21.15.155:6789/0 conn(0x7f648010aef0 legacy=0x7f64801a42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:49.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.080+0000 7f647cff9700 1 -- 172.21.15.43:0/563724797 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64801ac730 con 0x7f6480100960 2024-03-21T12:52:49.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.080+0000 7f6485f7a700 1 -- 172.21.15.43:0/563724797 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64801ab720 con 0x7f6480100960 2024-03-21T12:52:49.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.080+0000 7f6485f7a700 1 -- 172.21.15.43:0/563724797 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f64801abbe0 con 0x7f6480100960 2024-03-21T12:52:49.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.080+0000 7f647cff9700 1 -- 172.21.15.43:0/563724797 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f6470001ca0 con 0x7f6480100960 2024-03-21T12:52:49.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.080+0000 7f647cff9700 1 -- 172.21.15.43:0/563724797 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f6470005dd0 con 0x7f6480100960 2024-03-21T12:52:49.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.081+0000 7f6485f7a700 1 -- 172.21.15.43:0/563724797 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f646c005120 con 0x7f6480100960 2024-03-21T12:52:49.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.081+0000 7f647cff9700 1 -- 172.21.15.43:0/563724797 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7f647001c770 con 0x7f6480100960 2024-03-21T12:52:49.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.081+0000 7f647cff9700 1 -- 172.21.15.43:0/563724797 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 915400759 0 0) 0x7f64700899c0 con 0x7f6480100960 2024-03-21T12:52:49.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.086+0000 7f647cff9700 1 -- 172.21.15.43:0/563724797 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f6470056ff0 con 0x7f6480100960 2024-03-21T12:52:49.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.292+0000 7f6485f7a700 1 -- 172.21.15.43:0/563724797 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f646c005410 con 0x7f6480100960 2024-03-21T12:52:49.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.292+0000 7f647cff9700 1 -- 172.21.15.43:0/563724797 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (unknown 1274345170 0 3537426591) 0x7f647005aa40 con 0x7f6480100960 2024-03-21T12:52:49.293 INFO:teuthology.orchestra.run.smithi043.stdout:201863462934 2024-03-21T12:52:49.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.294+0000 7f6485f7a700 1 -- 172.21.15.43:0/563724797 >> v1:172.21.15.43:6800/4199387216 conn(0x7f646806f310 legacy=0x7f64680717d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:49.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.294+0000 7f6485f7a700 1 -- 172.21.15.43:0/563724797 >> v1:172.21.15.43:6789/0 conn(0x7f6480100960 legacy=0x7f6480107160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:49.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.294+0000 7f6485f7a700 1 -- 172.21.15.43:0/563724797 shutdown_connections 2024-03-21T12:52:49.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.294+0000 7f6485f7a700 1 -- 172.21.15.43:0/563724797 >> 172.21.15.43:0/563724797 conn(0x7f6480076180 msgr2=0x7f6480111f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:49.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.294+0000 7f6485f7a700 1 -- 172.21.15.43:0/563724797 shutdown_connections 2024-03-21T12:52:49.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:49.295+0000 7f6485f7a700 1 -- 172.21.15.43:0/563724797 wait complete. 2024-03-21T12:52:49.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:49 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:52:49.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:49 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1748581214' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-21T12:52:49.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:49 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/563724797' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-21T12:52:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:49 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:52:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:49 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1748581214' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-21T12:52:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:49 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/563724797' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-21T12:52:49.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:49 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:52:49.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:49 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1748581214' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-21T12:52:49.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:49 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/563724797' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-21T12:52:50.133 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149732 got 111669149735 for osd.3 2024-03-21T12:52:50.134 DEBUG:teuthology.parallel:result is None 2024-03-21T12:52:50.587 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411369 got 77309411372 for osd.2 2024-03-21T12:52:50.587 DEBUG:teuthology.parallel:result is None 2024-03-21T12:52:50.726 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:50 smithi043 ceph-mon[30062]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:52:50.726 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:50 smithi043 ceph-mon[41699]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:52:50.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:50 smithi155 ceph-mon[33846]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:52:51.119 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462931 got 201863462934 for osd.6 2024-03-21T12:52:51.119 DEBUG:teuthology.parallel:result is None 2024-03-21T12:52:51.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.379+0000 7f6398c48700 1 -- 172.21.15.43:0/1741362427 >> v1:172.21.15.43:6789/0 conn(0x7f639410e9d0 legacy=0x7f6394110e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:51.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.380+0000 7f6398c48700 1 -- 172.21.15.43:0/1741362427 shutdown_connections 2024-03-21T12:52:51.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.380+0000 7f6398c48700 1 -- 172.21.15.43:0/1741362427 >> 172.21.15.43:0/1741362427 conn(0x7f63940fda30 msgr2=0x7f63940ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:51.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.380+0000 7f6398c48700 1 -- 172.21.15.43:0/1741362427 shutdown_connections 2024-03-21T12:52:51.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.380+0000 7f6398c48700 1 -- 172.21.15.43:0/1741362427 wait complete. 2024-03-21T12:52:51.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.380+0000 7f6398c48700 1 Processor -- start 2024-03-21T12:52:51.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.380+0000 7f6398c48700 1 -- start start 2024-03-21T12:52:51.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.381+0000 7f6398c48700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f63941a9080 con 0x7f639410e9d0 2024-03-21T12:52:51.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.381+0000 7f6398c48700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f63941aa260 con 0x7f63941021a0 2024-03-21T12:52:51.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.381+0000 7f6398c48700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f63941ab480 con 0x7f639410ae30 2024-03-21T12:52:51.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.381+0000 7f6391d9b700 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7f639410ae30 0x7f63941a4230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:53696/0 (socket says 172.21.15.43:53696) 2024-03-21T12:52:51.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.381+0000 7f6391d9b700 1 -- 172.21.15.43:0/2433497377 learned_addr learned my addr 172.21.15.43:0/2433497377 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:52:51.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.381+0000 7f638b7fe700 1 -- 172.21.15.43:0/2433497377 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 362871658 0 0) 0x7f63941ab480 con 0x7f639410ae30 2024-03-21T12:52:51.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.381+0000 7f638b7fe700 1 -- 172.21.15.43:0/2433497377 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63800035b0 con 0x7f639410ae30 2024-03-21T12:52:51.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.381+0000 7f638b7fe700 1 -- 172.21.15.43:0/2433497377 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1931376231 0 0) 0x7f63941a9080 con 0x7f639410e9d0 2024-03-21T12:52:51.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.381+0000 7f638b7fe700 1 -- 172.21.15.43:0/2433497377 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63941ab480 con 0x7f639410e9d0 2024-03-21T12:52:51.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.381+0000 7f638b7fe700 1 -- 172.21.15.43:0/2433497377 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1095617518 0 0) 0x7f63941aa260 con 0x7f63941021a0 2024-03-21T12:52:51.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.381+0000 7f638b7fe700 1 -- 172.21.15.43:0/2433497377 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63941a9080 con 0x7f63941021a0 2024-03-21T12:52:51.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.381+0000 7f638b7fe700 1 -- 172.21.15.43:0/2433497377 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3842325316 0 0) 0x7f63800035b0 con 0x7f639410ae30 2024-03-21T12:52:51.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.381+0000 7f638b7fe700 1 -- 172.21.15.43:0/2433497377 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f63941aa260 con 0x7f639410ae30 2024-03-21T12:52:51.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.381+0000 7f638b7fe700 1 -- 172.21.15.43:0/2433497377 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f637c0019b0 con 0x7f639410ae30 2024-03-21T12:52:51.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.381+0000 7f638b7fe700 1 -- 172.21.15.43:0/2433497377 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2782412165 0 0) 0x7f63941ab480 con 0x7f639410e9d0 2024-03-21T12:52:51.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.381+0000 7f638b7fe700 1 -- 172.21.15.43:0/2433497377 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f63800035b0 con 0x7f639410e9d0 2024-03-21T12:52:51.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.381+0000 7f638b7fe700 1 -- 172.21.15.43:0/2433497377 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f638c001890 con 0x7f639410e9d0 2024-03-21T12:52:51.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.381+0000 7f638b7fe700 1 -- 172.21.15.43:0/2433497377 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1002294924 0 0) 0x7f63941a9080 con 0x7f63941021a0 2024-03-21T12:52:51.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.381+0000 7f638b7fe700 1 -- 172.21.15.43:0/2433497377 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f63941ab480 con 0x7f63941021a0 2024-03-21T12:52:51.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.381+0000 7f638b7fe700 1 -- 172.21.15.43:0/2433497377 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f63840017b0 con 0x7f63941021a0 2024-03-21T12:52:51.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.382+0000 7f638b7fe700 1 -- 172.21.15.43:0/2433497377 <== mon.2 v1:172.21.15.43:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2937532929 0 0) 0x7f63941aa260 con 0x7f639410ae30 2024-03-21T12:52:51.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.382+0000 7f638b7fe700 1 -- 172.21.15.43:0/2433497377 >> v1:172.21.15.155:6789/0 conn(0x7f63941021a0 legacy=0x7f6394107230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:51.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.382+0000 7f638b7fe700 1 -- 172.21.15.43:0/2433497377 >> v1:172.21.15.43:6789/0 conn(0x7f639410e9d0 legacy=0x7f63941a7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:51.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.382+0000 7f638b7fe700 1 -- 172.21.15.43:0/2433497377 --> v1:172.21.15.43:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63941ac6a0 con 0x7f639410ae30 2024-03-21T12:52:51.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.382+0000 7f6398c48700 1 -- 172.21.15.43:0/2433497377 --> v1:172.21.15.43:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63941aa490 con 0x7f639410ae30 2024-03-21T12:52:51.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.382+0000 7f6398c48700 1 -- 172.21.15.43:0/2433497377 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f63941aa950 con 0x7f639410ae30 2024-03-21T12:52:51.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.382+0000 7f638b7fe700 1 -- 172.21.15.43:0/2433497377 <== mon.2 v1:172.21.15.43:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f637c003b10 con 0x7f639410ae30 2024-03-21T12:52:51.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.382+0000 7f638b7fe700 1 -- 172.21.15.43:0/2433497377 <== mon.2 v1:172.21.15.43:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f637c005c30 con 0x7f639410ae30 2024-03-21T12:52:51.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.383+0000 7f6398c48700 1 -- 172.21.15.43:0/2433497377 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6374005120 con 0x7f639410ae30 2024-03-21T12:52:51.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.383+0000 7f638b7fe700 1 -- 172.21.15.43:0/2433497377 <== mon.2 v1:172.21.15.43:6790/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7f637c003680 con 0x7f639410ae30 2024-03-21T12:52:51.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.384+0000 7f638b7fe700 1 -- 172.21.15.43:0/2433497377 <== mon.2 v1:172.21.15.43:6790/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 915400759 0 0) 0x7f637c0898e0 con 0x7f639410ae30 2024-03-21T12:52:51.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.388+0000 7f638b7fe700 1 -- 172.21.15.43:0/2433497377 <== mon.2 v1:172.21.15.43:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f637c056eb0 con 0x7f639410ae30 2024-03-21T12:52:51.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.596+0000 7f6398c48700 1 -- 172.21.15.43:0/2433497377 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f6374005410 con 0x7f639410ae30 2024-03-21T12:52:51.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.596+0000 7f638b7fe700 1 -- 172.21.15.43:0/2433497377 <== mon.2 v1:172.21.15.43:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (unknown 1482059429 0 397825816) 0x7f637c05a900 con 0x7f639410ae30 2024-03-21T12:52:51.596 INFO:teuthology.orchestra.run.smithi043.stdout:236223201299 2024-03-21T12:52:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.598+0000 7f6398c48700 1 -- 172.21.15.43:0/2433497377 >> v1:172.21.15.43:6800/4199387216 conn(0x7f638006f230 legacy=0x7f63800716f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.598+0000 7f6398c48700 1 -- 172.21.15.43:0/2433497377 >> v1:172.21.15.43:6790/0 conn(0x7f639410ae30 legacy=0x7f63941a4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:51.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.598+0000 7f6398c48700 1 -- 172.21.15.43:0/2433497377 shutdown_connections 2024-03-21T12:52:51.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.598+0000 7f6398c48700 1 -- 172.21.15.43:0/2433497377 >> 172.21.15.43:0/2433497377 conn(0x7f63940fda30 msgr2=0x7f63940fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:51.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.599+0000 7f6398c48700 1 -- 172.21.15.43:0/2433497377 shutdown_connections 2024-03-21T12:52:51.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:51.599+0000 7f6398c48700 1 -- 172.21.15.43:0/2433497377 wait complete. 2024-03-21T12:52:52.252 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201294 got 236223201299 for osd.7 2024-03-21T12:52:52.253 DEBUG:teuthology.parallel:result is None 2024-03-21T12:52:52.253 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-21T12:52:52.253 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-21T12:52:52.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:52 smithi043 ceph-mon[30062]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:52:52.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:52 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:52.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:52 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:52.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2433497377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-21T12:52:52.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:52 smithi043 ceph-mon[41699]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:52:52.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:52 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:52.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:52 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:52.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2433497377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-21T12:52:52.500 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:52:52.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:52 smithi155 ceph-mon[33846]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:52:52.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:52 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:52.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:52 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:52.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2433497377' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-21T12:52:53.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:53.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:53.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:53.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:53.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:53.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:53.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:53.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:53.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:53.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:53.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[41699]: Adjusting osd_memory_target on smithi043 to 3761M 2024-03-21T12:52:53.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:53.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:52:53.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:52:53.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[41699]: Updating smithi043:/etc/ceph/ceph.conf 2024-03-21T12:52:53.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[41699]: Updating smithi155:/etc/ceph/ceph.conf 2024-03-21T12:52:53.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:53.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:53.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:53.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:53.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:53.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:53.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:53.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:53.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:53.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:53.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:53.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:53.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[30062]: Adjusting osd_memory_target on smithi043 to 3761M 2024-03-21T12:52:53.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:53.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:52:53.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:52:53.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[30062]: Updating smithi043:/etc/ceph/ceph.conf 2024-03-21T12:52:53.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[30062]: Updating smithi155:/etc/ceph/ceph.conf 2024-03-21T12:52:53.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:53.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:53 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:53.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:53 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:53.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:53 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:53.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:53 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:53.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:53 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:53.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:53 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:53.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:53 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:53.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:53 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:53.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:53 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:53.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:53 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:53.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:53 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T12:52:53.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:53 smithi155 ceph-mon[33846]: Adjusting osd_memory_target on smithi043 to 3761M 2024-03-21T12:52:53.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:53 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:53.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:53 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:52:53.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:53 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:52:53.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:53 smithi155 ceph-mon[33846]: Updating smithi043:/etc/ceph/ceph.conf 2024-03-21T12:52:53.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:53 smithi155 ceph-mon[33846]: Updating smithi155:/etc/ceph/ceph.conf 2024-03-21T12:52:53.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:53 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:53.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:53 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:54.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.334+0000 7f423a033700 1 -- 172.21.15.43:0/1041732865 >> v1:172.21.15.43:6790/0 conn(0x7f4234106fb0 legacy=0x7f4234107390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:54.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.334+0000 7f423a033700 1 -- 172.21.15.43:0/1041732865 shutdown_connections 2024-03-21T12:52:54.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.334+0000 7f423a033700 1 -- 172.21.15.43:0/1041732865 >> 172.21.15.43:0/1041732865 conn(0x7f4234076120 msgr2=0x7f4234076530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:54.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.334+0000 7f423a033700 1 -- 172.21.15.43:0/1041732865 shutdown_connections 2024-03-21T12:52:54.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.334+0000 7f423a033700 1 -- 172.21.15.43:0/1041732865 wait complete. 2024-03-21T12:52:54.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.334+0000 7f423a033700 1 Processor -- start 2024-03-21T12:52:54.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.335+0000 7f423a033700 1 -- start start 2024-03-21T12:52:54.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.335+0000 7f423a033700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f42341a7100 con 0x7f4234106fb0 2024-03-21T12:52:54.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.335+0000 7f423a033700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f42341a8320 con 0x7f423410ea80 2024-03-21T12:52:54.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.335+0000 7f423a033700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f42341a9560 con 0x7f423410ade0 2024-03-21T12:52:54.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.335+0000 7f4232ffd700 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7f423410ade0 0x7f42341a1ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:40856/0 (socket says 172.21.15.43:40856) 2024-03-21T12:52:54.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.335+0000 7f4232ffd700 1 -- 172.21.15.43:0/1788132522 learned_addr learned my addr 172.21.15.43:0/1788132522 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:52:54.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.335+0000 7f4230ff9700 1 -- 172.21.15.43:0/1788132522 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 132628305 0 0) 0x7f42341a7100 con 0x7f4234106fb0 2024-03-21T12:52:54.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.335+0000 7f4230ff9700 1 -- 172.21.15.43:0/1788132522 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f421c0035b0 con 0x7f4234106fb0 2024-03-21T12:52:54.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.335+0000 7f4230ff9700 1 -- 172.21.15.43:0/1788132522 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1573134838 0 0) 0x7f42341a9560 con 0x7f423410ade0 2024-03-21T12:52:54.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.335+0000 7f4230ff9700 1 -- 172.21.15.43:0/1788132522 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42341a7100 con 0x7f423410ade0 2024-03-21T12:52:54.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.335+0000 7f4230ff9700 1 -- 172.21.15.43:0/1788132522 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1545679301 0 0) 0x7f42341a8320 con 0x7f423410ea80 2024-03-21T12:52:54.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.335+0000 7f4230ff9700 1 -- 172.21.15.43:0/1788132522 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42341a9560 con 0x7f423410ea80 2024-03-21T12:52:54.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.335+0000 7f4230ff9700 1 -- 172.21.15.43:0/1788132522 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 300494327 0 0) 0x7f421c0035b0 con 0x7f4234106fb0 2024-03-21T12:52:54.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.335+0000 7f4230ff9700 1 -- 172.21.15.43:0/1788132522 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f42341a8320 con 0x7f4234106fb0 2024-03-21T12:52:54.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.335+0000 7f4230ff9700 1 -- 172.21.15.43:0/1788132522 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f4224001880 con 0x7f4234106fb0 2024-03-21T12:52:54.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.335+0000 7f4230ff9700 1 -- 172.21.15.43:0/1788132522 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2184662257 0 0) 0x7f42341a7100 con 0x7f423410ade0 2024-03-21T12:52:54.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.335+0000 7f4230ff9700 1 -- 172.21.15.43:0/1788132522 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f421c0035b0 con 0x7f423410ade0 2024-03-21T12:52:54.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.335+0000 7f4230ff9700 1 -- 172.21.15.43:0/1788132522 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f42280017b0 con 0x7f423410ade0 2024-03-21T12:52:54.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.335+0000 7f4230ff9700 1 -- 172.21.15.43:0/1788132522 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1515656516 0 0) 0x7f42341a9560 con 0x7f423410ea80 2024-03-21T12:52:54.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.335+0000 7f4230ff9700 1 -- 172.21.15.43:0/1788132522 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f42341a7100 con 0x7f423410ea80 2024-03-21T12:52:54.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.336+0000 7f4230ff9700 1 -- 172.21.15.43:0/1788132522 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f422c001c50 con 0x7f423410ea80 2024-03-21T12:52:54.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.336+0000 7f4230ff9700 1 -- 172.21.15.43:0/1788132522 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3699463816 0 0) 0x7f42341a8320 con 0x7f4234106fb0 2024-03-21T12:52:54.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.336+0000 7f4230ff9700 1 -- 172.21.15.43:0/1788132522 >> v1:172.21.15.43:6790/0 conn(0x7f423410ade0 legacy=0x7f42341a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:54.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.336+0000 7f4230ff9700 1 -- 172.21.15.43:0/1788132522 >> v1:172.21.15.155:6789/0 conn(0x7f423410ea80 legacy=0x7f42341a57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:54.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.336+0000 7f4230ff9700 1 -- 172.21.15.43:0/1788132522 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42341aa7a0 con 0x7f4234106fb0 2024-03-21T12:52:54.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.336+0000 7f423a033700 1 -- 172.21.15.43:0/1788132522 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42341a9790 con 0x7f4234106fb0 2024-03-21T12:52:54.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.336+0000 7f423a033700 1 -- 172.21.15.43:0/1788132522 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f42341a9c50 con 0x7f4234106fb0 2024-03-21T12:52:54.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.336+0000 7f4230ff9700 1 -- 172.21.15.43:0/1788132522 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f4224004770 con 0x7f4234106fb0 2024-03-21T12:52:54.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.336+0000 7f4230ff9700 1 -- 172.21.15.43:0/1788132522 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f4224004d30 con 0x7f4234106fb0 2024-03-21T12:52:54.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.336+0000 7f423a033700 1 -- 172.21.15.43:0/1788132522 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4220005120 con 0x7f4234106fb0 2024-03-21T12:52:54.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.337+0000 7f4230ff9700 1 -- 172.21.15.43:0/1788132522 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7f4224001b30 con 0x7f4234106fb0 2024-03-21T12:52:54.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.338+0000 7f4230ff9700 1 -- 172.21.15.43:0/1788132522 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 915400759 0 0) 0x7f4224088620 con 0x7f4234106fb0 2024-03-21T12:52:54.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.342+0000 7f4230ff9700 1 -- 172.21.15.43:0/1788132522 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f4224055c50 con 0x7f4234106fb0 2024-03-21T12:52:54.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.552+0000 7f423a033700 1 -- 172.21.15.43:0/1788132522 --> v1:172.21.15.43:6800/4199387216 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4220000b90 con 0x7f421c06f180 2024-03-21T12:52:54.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.564+0000 7f4230ff9700 1 -- 172.21.15.43:0/1788132522 <== mgr.34493 v1:172.21.15.43:6800/4199387216 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+346466 (unknown 2965378022 0 2534608281) 0x7f4220000b90 con 0x7f421c06f180 2024-03-21T12:52:54.573 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:52:54.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.569+0000 7f423a033700 1 -- 172.21.15.43:0/1788132522 >> v1:172.21.15.43:6800/4199387216 conn(0x7f421c06f180 legacy=0x7f421c071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:54.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.569+0000 7f423a033700 1 -- 172.21.15.43:0/1788132522 >> v1:172.21.15.43:6789/0 conn(0x7f4234106fb0 legacy=0x7f4234101040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:54.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.569+0000 7f423a033700 1 -- 172.21.15.43:0/1788132522 shutdown_connections 2024-03-21T12:52:54.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.569+0000 7f423a033700 1 -- 172.21.15.43:0/1788132522 >> 172.21.15.43:0/1788132522 conn(0x7f4234076120 msgr2=0x7f42341116b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:54.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.569+0000 7f423a033700 1 -- 172.21.15.43:0/1788132522 shutdown_connections 2024-03-21T12:52:54.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:54.569+0000 7f423a033700 1 -- 172.21.15.43:0/1788132522 wait complete. 2024-03-21T12:52:54.578 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-03-21T12:52:54.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:54 smithi043 ceph-mon[30062]: Updating smithi043:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T12:52:54.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:54 smithi043 ceph-mon[30062]: Updating smithi155:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T12:52:54.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:54 smithi043 ceph-mon[30062]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-03-21T12:52:54.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:54 smithi043 ceph-mon[30062]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:52:54.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:54 smithi043 ceph-mon[30062]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-03-21T12:52:54.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:54 smithi043 ceph-mon[30062]: Updating smithi043:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T12:52:54.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:54 smithi043 ceph-mon[30062]: Updating smithi155:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T12:52:54.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:54 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:54.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:54 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:54.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:54 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:54.893 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:54 smithi043 ceph-mon[30062]: Deploying daemon alertmanager.a on smithi043 2024-03-21T12:52:54.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:54 smithi043 ceph-mon[41699]: Updating smithi043:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T12:52:54.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:54 smithi043 ceph-mon[41699]: Updating smithi155:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T12:52:54.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:54 smithi043 ceph-mon[41699]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-03-21T12:52:54.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:54 smithi043 ceph-mon[41699]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:52:54.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:54 smithi043 ceph-mon[41699]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-03-21T12:52:54.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:54 smithi043 ceph-mon[41699]: Updating smithi043:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T12:52:54.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:54 smithi043 ceph-mon[41699]: Updating smithi155:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T12:52:54.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:54 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:54.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:54 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:54.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:54 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:54.894 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:54 smithi043 ceph-mon[41699]: Deploying daemon alertmanager.a on smithi043 2024-03-21T12:52:54.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:54 smithi155 ceph-mon[33846]: Updating smithi043:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T12:52:54.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:54 smithi155 ceph-mon[33846]: Updating smithi155:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T12:52:54.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:54 smithi155 ceph-mon[33846]: Updating smithi043:/etc/ceph/ceph.client.admin.keyring 2024-03-21T12:52:54.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:54 smithi155 ceph-mon[33846]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:52:54.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:54 smithi155 ceph-mon[33846]: Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-03-21T12:52:54.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:54 smithi155 ceph-mon[33846]: Updating smithi043:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T12:52:54.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:54 smithi155 ceph-mon[33846]: Updating smithi155:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T12:52:54.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:54 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:54.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:54 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:54.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:54 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:52:54.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:54 smithi155 ceph-mon[33846]: Deploying daemon alertmanager.a on smithi043 2024-03-21T12:52:55.523 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-03-21T12:52:53.067546+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596392,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1005,"num_read_kb":851,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1771434,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":236084,"kb_used_data":7452,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":749725132,"statfs":{"total":767960285184,"available":767718535168,"internally_reserved":0,"allocated":7630848,"data_stored":4046672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003230"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051825+0000","last_change":"2024-03-21T12:51:42.088178+0000","last_active":"2024-03-21T12:52:17.051825+0000","last_peered":"2024-03-21T12:52:17.051825+0000","last_clean":"2024-03-21T12:52:17.051825+0000","last_became_active":"2024-03-21T12:51:42.087895+0000","last_became_peered":"2024-03-21T12:51:42.087895+0000","last_unstale":"2024-03-21T12:52:17.051825+0000","last_undegraded":"2024-03-21T12:52:17.051825+0000","last_fullsized":"2024-03-21T12:52:17.051825+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:07:06.943938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179443+0000","last_change":"2024-03-21T12:51:40.091222+0000","last_active":"2024-03-21T12:52:17.179443+0000","last_peered":"2024-03-21T12:52:17.179443+0000","last_clean":"2024-03-21T12:52:17.179443+0000","last_became_active":"2024-03-21T12:51:40.089461+0000","last_became_peered":"2024-03-21T12:51:40.089461+0000","last_unstale":"2024-03-21T12:52:17.179443+0000","last_undegraded":"2024-03-21T12:52:17.179443+0000","last_fullsized":"2024-03-21T12:52:17.179443+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:21:07.377799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180175+0000","last_change":"2024-03-21T12:51:36.075348+0000","last_active":"2024-03-21T12:52:17.180175+0000","last_peered":"2024-03-21T12:52:17.180175+0000","last_clean":"2024-03-21T12:52:17.180175+0000","last_became_active":"2024-03-21T12:51:36.075007+0000","last_became_peered":"2024-03-21T12:51:36.075007+0000","last_unstale":"2024-03-21T12:52:17.180175+0000","last_undegraded":"2024-03-21T12:52:17.180175+0000","last_fullsized":"2024-03-21T12:52:17.180175+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:19:54.724289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"69'9","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051764+0000","last_change":"2024-03-21T12:51:38.088822+0000","last_active":"2024-03-21T12:52:17.051764+0000","last_peered":"2024-03-21T12:52:17.051764+0000","last_clean":"2024-03-21T12:52:17.051764+0000","last_became_active":"2024-03-21T12:51:38.088084+0000","last_became_peered":"2024-03-21T12:51:38.088084+0000","last_unstale":"2024-03-21T12:52:17.051764+0000","last_undegraded":"2024-03-21T12:52:17.051764+0000","last_fullsized":"2024-03-21T12:52:17.051764+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:13:44.640011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.050866+0000","last_change":"2024-03-21T12:51:42.181273+0000","last_active":"2024-03-21T12:52:17.050866+0000","last_peered":"2024-03-21T12:52:17.050866+0000","last_clean":"2024-03-21T12:52:17.050866+0000","last_became_active":"2024-03-21T12:51:42.180885+0000","last_became_peered":"2024-03-21T12:51:42.180885+0000","last_unstale":"2024-03-21T12:52:17.050866+0000","last_undegraded":"2024-03-21T12:52:17.050866+0000","last_fullsized":"2024-03-21T12:52:17.050866+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:35:57.826722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051771+0000","last_change":"2024-03-21T12:51:40.087635+0000","last_active":"2024-03-21T12:52:17.051771+0000","last_peered":"2024-03-21T12:52:17.051771+0000","last_clean":"2024-03-21T12:52:17.051771+0000","last_became_active":"2024-03-21T12:51:40.087342+0000","last_became_peered":"2024-03-21T12:51:40.087342+0000","last_unstale":"2024-03-21T12:52:17.051771+0000","last_undegraded":"2024-03-21T12:52:17.051771+0000","last_fullsized":"2024-03-21T12:52:17.051771+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:39:55.056529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051378+0000","last_change":"2024-03-21T12:51:36.073811+0000","last_active":"2024-03-21T12:52:17.051378+0000","last_peered":"2024-03-21T12:52:17.051378+0000","last_clean":"2024-03-21T12:52:17.051378+0000","last_became_active":"2024-03-21T12:51:36.073489+0000","last_became_peered":"2024-03-21T12:51:36.073489+0000","last_unstale":"2024-03-21T12:52:17.051378+0000","last_undegraded":"2024-03-21T12:52:17.051378+0000","last_fullsized":"2024-03-21T12:52:17.051378+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:33:48.629162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"69'15","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.050775+0000","last_change":"2024-03-21T12:51:38.081956+0000","last_active":"2024-03-21T12:52:17.050775+0000","last_peered":"2024-03-21T12:52:17.050775+0000","last_clean":"2024-03-21T12:52:17.050775+0000","last_became_active":"2024-03-21T12:51:38.081141+0000","last_became_peered":"2024-03-21T12:51:38.081141+0000","last_unstale":"2024-03-21T12:52:17.050775+0000","last_undegraded":"2024-03-21T12:52:17.050775+0000","last_fullsized":"2024-03-21T12:52:17.050775+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:27:55.935543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180339+0000","last_change":"2024-03-21T12:51:42.097783+0000","last_active":"2024-03-21T12:52:17.180339+0000","last_peered":"2024-03-21T12:52:17.180339+0000","last_clean":"2024-03-21T12:52:17.180339+0000","last_became_active":"2024-03-21T12:51:42.097376+0000","last_became_peered":"2024-03-21T12:51:42.097376+0000","last_unstale":"2024-03-21T12:52:17.180339+0000","last_undegraded":"2024-03-21T12:52:17.180339+0000","last_fullsized":"2024-03-21T12:52:17.180339+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:53:25.723317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157238+0000","last_change":"2024-03-21T12:51:40.094677+0000","last_active":"2024-03-21T12:52:17.157238+0000","last_peered":"2024-03-21T12:52:17.157238+0000","last_clean":"2024-03-21T12:52:17.157238+0000","last_became_active":"2024-03-21T12:51:40.094278+0000","last_became_peered":"2024-03-21T12:51:40.094278+0000","last_unstale":"2024-03-21T12:52:17.157238+0000","last_undegraded":"2024-03-21T12:52:17.157238+0000","last_fullsized":"2024-03-21T12:52:17.157238+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:07:38.302277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157565+0000","last_change":"2024-03-21T12:51:36.073879+0000","last_active":"2024-03-21T12:52:17.157565+0000","last_peered":"2024-03-21T12:52:17.157565+0000","last_clean":"2024-03-21T12:52:17.157565+0000","last_became_active":"2024-03-21T12:51:36.073590+0000","last_became_peered":"2024-03-21T12:51:36.073590+0000","last_unstale":"2024-03-21T12:52:17.157565+0000","last_undegraded":"2024-03-21T12:52:17.157565+0000","last_fullsized":"2024-03-21T12:52:17.157565+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:57:35.709674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"69'9","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.052376+0000","last_change":"2024-03-21T12:51:38.084906+0000","last_active":"2024-03-21T12:52:17.052376+0000","last_peered":"2024-03-21T12:52:17.052376+0000","last_clean":"2024-03-21T12:52:17.052376+0000","last_became_active":"2024-03-21T12:51:38.084472+0000","last_became_peered":"2024-03-21T12:51:38.084472+0000","last_unstale":"2024-03-21T12:52:17.052376+0000","last_undegraded":"2024-03-21T12:52:17.052376+0000","last_fullsized":"2024-03-21T12:52:17.052376+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:10:54.770995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.156919+0000","last_change":"2024-03-21T12:51:42.087206+0000","last_active":"2024-03-21T12:52:17.156919+0000","last_peered":"2024-03-21T12:52:17.156919+0000","last_clean":"2024-03-21T12:52:17.156919+0000","last_became_active":"2024-03-21T12:51:42.086771+0000","last_became_peered":"2024-03-21T12:51:42.086771+0000","last_unstale":"2024-03-21T12:52:17.156919+0000","last_undegraded":"2024-03-21T12:52:17.156919+0000","last_fullsized":"2024-03-21T12:52:17.156919+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:40:35.449173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.163378+0000","last_change":"2024-03-21T12:51:40.092051+0000","last_active":"2024-03-21T12:52:17.163378+0000","last_peered":"2024-03-21T12:52:17.163378+0000","last_clean":"2024-03-21T12:52:17.163378+0000","last_became_active":"2024-03-21T12:51:40.091783+0000","last_became_peered":"2024-03-21T12:51:40.091783+0000","last_unstale":"2024-03-21T12:52:17.163378+0000","last_undegraded":"2024-03-21T12:52:17.163378+0000","last_fullsized":"2024-03-21T12:52:17.163378+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:47:33.547754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157294+0000","last_change":"2024-03-21T12:51:36.068971+0000","last_active":"2024-03-21T12:52:17.157294+0000","last_peered":"2024-03-21T12:52:17.157294+0000","last_clean":"2024-03-21T12:52:17.157294+0000","last_became_active":"2024-03-21T12:51:36.068652+0000","last_became_peered":"2024-03-21T12:51:36.068652+0000","last_unstale":"2024-03-21T12:52:17.157294+0000","last_undegraded":"2024-03-21T12:52:17.157294+0000","last_fullsized":"2024-03-21T12:52:17.157294+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:25:21.661020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"69'5","reported_seq":39,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.052212+0000","last_change":"2024-03-21T12:51:38.088696+0000","last_active":"2024-03-21T12:52:17.052212+0000","last_peered":"2024-03-21T12:52:17.052212+0000","last_clean":"2024-03-21T12:52:17.052212+0000","last_became_active":"2024-03-21T12:51:38.088249+0000","last_became_peered":"2024-03-21T12:51:38.088249+0000","last_unstale":"2024-03-21T12:52:17.052212+0000","last_undegraded":"2024-03-21T12:52:17.052212+0000","last_fullsized":"2024-03-21T12:52:17.052212+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:21:23.040367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.053471+0000","last_change":"2024-03-21T12:51:42.179297+0000","last_active":"2024-03-21T12:52:17.053471+0000","last_peered":"2024-03-21T12:52:17.053471+0000","last_clean":"2024-03-21T12:52:17.053471+0000","last_became_active":"2024-03-21T12:51:42.178990+0000","last_became_peered":"2024-03-21T12:51:42.178990+0000","last_unstale":"2024-03-21T12:52:17.053471+0000","last_undegraded":"2024-03-21T12:52:17.053471+0000","last_fullsized":"2024-03-21T12:52:17.053471+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:49:36.111327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180469+0000","last_change":"2024-03-21T12:51:40.084280+0000","last_active":"2024-03-21T12:52:17.180469+0000","last_peered":"2024-03-21T12:52:17.180469+0000","last_clean":"2024-03-21T12:52:17.180469+0000","last_became_active":"2024-03-21T12:51:40.083940+0000","last_became_peered":"2024-03-21T12:51:40.083940+0000","last_unstale":"2024-03-21T12:52:17.180469+0000","last_undegraded":"2024-03-21T12:52:17.180469+0000","last_fullsized":"2024-03-21T12:52:17.180469+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:48:19.807821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180896+0000","last_change":"2024-03-21T12:51:36.073926+0000","last_active":"2024-03-21T12:52:17.180896+0000","last_peered":"2024-03-21T12:52:17.180896+0000","last_clean":"2024-03-21T12:52:17.180896+0000","last_became_active":"2024-03-21T12:51:36.073613+0000","last_became_peered":"2024-03-21T12:51:36.073613+0000","last_unstale":"2024-03-21T12:52:17.180896+0000","last_undegraded":"2024-03-21T12:52:17.180896+0000","last_fullsized":"2024-03-21T12:52:17.180896+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:01:12.322618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"69'15","reported_seq":76,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.049949+0000","last_change":"2024-03-21T12:51:53.006013+0000","last_active":"2024-03-21T12:52:17.049949+0000","last_peered":"2024-03-21T12:52:17.049949+0000","last_clean":"2024-03-21T12:52:17.049949+0000","last_became_active":"2024-03-21T12:51:52.993106+0000","last_became_peered":"2024-03-21T12:51:52.993106+0000","last_unstale":"2024-03-21T12:52:17.049949+0000","last_undegraded":"2024-03-21T12:52:17.049949+0000","last_fullsized":"2024-03-21T12:52:17.049949+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:56:15.803027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051713+0000","last_change":"2024-03-21T12:51:42.179543+0000","last_active":"2024-03-21T12:52:17.051713+0000","last_peered":"2024-03-21T12:52:17.051713+0000","last_clean":"2024-03-21T12:52:17.051713+0000","last_became_active":"2024-03-21T12:51:42.179167+0000","last_became_peered":"2024-03-21T12:51:42.179167+0000","last_unstale":"2024-03-21T12:52:17.051713+0000","last_undegraded":"2024-03-21T12:52:17.051713+0000","last_fullsized":"2024-03-21T12:52:17.051713+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:36:00.727087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.052210+0000","last_change":"2024-03-21T12:51:40.090061+0000","last_active":"2024-03-21T12:52:17.052210+0000","last_peered":"2024-03-21T12:52:17.052210+0000","last_clean":"2024-03-21T12:52:17.052210+0000","last_became_active":"2024-03-21T12:51:40.089710+0000","last_became_peered":"2024-03-21T12:51:40.089710+0000","last_unstale":"2024-03-21T12:52:17.052210+0000","last_undegraded":"2024-03-21T12:52:17.052210+0000","last_fullsized":"2024-03-21T12:52:17.052210+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:33:59.033849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.052425+0000","last_change":"2024-03-21T12:51:36.077115+0000","last_active":"2024-03-21T12:52:17.052425+0000","last_peered":"2024-03-21T12:52:17.052425+0000","last_clean":"2024-03-21T12:52:17.052425+0000","last_became_active":"2024-03-21T12:51:36.076799+0000","last_became_peered":"2024-03-21T12:51:36.076799+0000","last_unstale":"2024-03-21T12:52:17.052425+0000","last_undegraded":"2024-03-21T12:52:17.052425+0000","last_fullsized":"2024-03-21T12:52:17.052425+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:01:43.648779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"69'12","reported_seq":52,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051170+0000","last_change":"2024-03-21T12:51:38.084344+0000","last_active":"2024-03-21T12:52:17.051170+0000","last_peered":"2024-03-21T12:52:17.051170+0000","last_clean":"2024-03-21T12:52:17.051170+0000","last_became_active":"2024-03-21T12:51:38.084029+0000","last_became_peered":"2024-03-21T12:51:38.084029+0000","last_unstale":"2024-03-21T12:52:17.051170+0000","last_undegraded":"2024-03-21T12:52:17.051170+0000","last_fullsized":"2024-03-21T12:52:17.051170+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:55:38.851456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"69'1","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.163583+0000","last_change":"2024-03-21T12:51:42.094307+0000","last_active":"2024-03-21T12:52:17.163583+0000","last_peered":"2024-03-21T12:52:17.163583+0000","last_clean":"2024-03-21T12:52:17.163583+0000","last_became_active":"2024-03-21T12:51:42.093994+0000","last_became_peered":"2024-03-21T12:51:42.093994+0000","last_unstale":"2024-03-21T12:52:17.163583+0000","last_undegraded":"2024-03-21T12:52:17.163583+0000","last_fullsized":"2024-03-21T12:52:17.163583+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:13:57.757683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":370,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"69'8","reported_seq":52,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:46.131384+0000","last_change":"2024-03-21T12:51:40.087776+0000","last_active":"2024-03-21T12:52:46.131384+0000","last_peered":"2024-03-21T12:52:46.131384+0000","last_clean":"2024-03-21T12:52:46.131384+0000","last_became_active":"2024-03-21T12:51:40.087457+0000","last_became_peered":"2024-03-21T12:51:40.087457+0000","last_unstale":"2024-03-21T12:52:46.131384+0000","last_undegraded":"2024-03-21T12:52:46.131384+0000","last_fullsized":"2024-03-21T12:52:46.131384+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:13:56.709428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"62'1","reported_seq":34,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179514+0000","last_change":"2024-03-21T12:51:36.070247+0000","last_active":"2024-03-21T12:52:17.179514+0000","last_peered":"2024-03-21T12:52:17.179514+0000","last_clean":"2024-03-21T12:52:17.179514+0000","last_became_active":"2024-03-21T12:51:36.069956+0000","last_became_peered":"2024-03-21T12:51:36.069956+0000","last_unstale":"2024-03-21T12:52:17.179514+0000","last_undegraded":"2024-03-21T12:52:17.179514+0000","last_fullsized":"2024-03-21T12:52:17.179514+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:31:35.087019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"69'10","reported_seq":44,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180649+0000","last_change":"2024-03-21T12:51:38.217798+0000","last_active":"2024-03-21T12:52:17.180649+0000","last_peered":"2024-03-21T12:52:17.180649+0000","last_clean":"2024-03-21T12:52:17.180649+0000","last_became_active":"2024-03-21T12:51:38.217467+0000","last_became_peered":"2024-03-21T12:51:38.217467+0000","last_unstale":"2024-03-21T12:52:17.180649+0000","last_undegraded":"2024-03-21T12:52:17.180649+0000","last_fullsized":"2024-03-21T12:52:17.180649+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:13:10.608259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180625+0000","last_change":"2024-03-21T12:51:42.084785+0000","last_active":"2024-03-21T12:52:17.180625+0000","last_peered":"2024-03-21T12:52:17.180625+0000","last_clean":"2024-03-21T12:52:17.180625+0000","last_became_active":"2024-03-21T12:51:42.084320+0000","last_became_peered":"2024-03-21T12:51:42.084320+0000","last_unstale":"2024-03-21T12:52:17.180625+0000","last_undegraded":"2024-03-21T12:52:17.180625+0000","last_fullsized":"2024-03-21T12:52:17.180625+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:45:32.051347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180977+0000","last_change":"2024-03-21T12:51:40.087946+0000","last_active":"2024-03-21T12:52:17.180977+0000","last_peered":"2024-03-21T12:52:17.180977+0000","last_clean":"2024-03-21T12:52:17.180977+0000","last_became_active":"2024-03-21T12:51:40.087646+0000","last_became_peered":"2024-03-21T12:51:40.087646+0000","last_unstale":"2024-03-21T12:52:17.180977+0000","last_undegraded":"2024-03-21T12:52:17.180977+0000","last_fullsized":"2024-03-21T12:52:17.180977+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:39:08.769765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.049710+0000","last_change":"2024-03-21T12:51:36.074573+0000","last_active":"2024-03-21T12:52:17.049710+0000","last_peered":"2024-03-21T12:52:17.049710+0000","last_clean":"2024-03-21T12:52:17.049710+0000","last_became_active":"2024-03-21T12:51:36.073984+0000","last_became_peered":"2024-03-21T12:51:36.073984+0000","last_unstale":"2024-03-21T12:52:17.049710+0000","last_undegraded":"2024-03-21T12:52:17.049710+0000","last_fullsized":"2024-03-21T12:52:17.049710+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:22:44.877000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"69'11","reported_seq":48,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180776+0000","last_change":"2024-03-21T12:51:38.092551+0000","last_active":"2024-03-21T12:52:17.180776+0000","last_peered":"2024-03-21T12:52:17.180776+0000","last_clean":"2024-03-21T12:52:17.180776+0000","last_became_active":"2024-03-21T12:51:38.092200+0000","last_became_peered":"2024-03-21T12:51:38.092200+0000","last_unstale":"2024-03-21T12:52:17.180776+0000","last_undegraded":"2024-03-21T12:52:17.180776+0000","last_fullsized":"2024-03-21T12:52:17.180776+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:03:40.189935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.163345+0000","last_change":"2024-03-21T12:51:42.091573+0000","last_active":"2024-03-21T12:52:17.163345+0000","last_peered":"2024-03-21T12:52:17.163345+0000","last_clean":"2024-03-21T12:52:17.163345+0000","last_became_active":"2024-03-21T12:51:42.091194+0000","last_became_peered":"2024-03-21T12:51:42.091194+0000","last_unstale":"2024-03-21T12:52:17.163345+0000","last_undegraded":"2024-03-21T12:52:17.163345+0000","last_fullsized":"2024-03-21T12:52:17.163345+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:34:24.166192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179961+0000","last_change":"2024-03-21T12:51:40.087634+0000","last_active":"2024-03-21T12:52:17.179961+0000","last_peered":"2024-03-21T12:52:17.179961+0000","last_clean":"2024-03-21T12:52:17.179961+0000","last_became_active":"2024-03-21T12:51:40.087331+0000","last_became_peered":"2024-03-21T12:51:40.087331+0000","last_unstale":"2024-03-21T12:52:17.179961+0000","last_undegraded":"2024-03-21T12:52:17.179961+0000","last_fullsized":"2024-03-21T12:52:17.179961+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:58:54.841811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180074+0000","last_change":"2024-03-21T12:51:36.078841+0000","last_active":"2024-03-21T12:52:17.180074+0000","last_peered":"2024-03-21T12:52:17.180074+0000","last_clean":"2024-03-21T12:52:17.180074+0000","last_became_active":"2024-03-21T12:51:36.078533+0000","last_became_peered":"2024-03-21T12:51:36.078533+0000","last_unstale":"2024-03-21T12:52:17.180074+0000","last_undegraded":"2024-03-21T12:52:17.180074+0000","last_fullsized":"2024-03-21T12:52:17.180074+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:39:05.360528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"69'4","reported_seq":35,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.049814+0000","last_change":"2024-03-21T12:51:38.087208+0000","last_active":"2024-03-21T12:52:17.049814+0000","last_peered":"2024-03-21T12:52:17.049814+0000","last_clean":"2024-03-21T12:52:17.049814+0000","last_became_active":"2024-03-21T12:51:38.086879+0000","last_became_peered":"2024-03-21T12:51:38.086879+0000","last_unstale":"2024-03-21T12:52:17.049814+0000","last_undegraded":"2024-03-21T12:52:17.049814+0000","last_fullsized":"2024-03-21T12:52:17.049814+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:56:24.378008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.050668+0000","last_change":"2024-03-21T12:51:42.180358+0000","last_active":"2024-03-21T12:52:17.050668+0000","last_peered":"2024-03-21T12:52:17.050668+0000","last_clean":"2024-03-21T12:52:17.050668+0000","last_became_active":"2024-03-21T12:51:42.180036+0000","last_became_peered":"2024-03-21T12:51:42.180036+0000","last_unstale":"2024-03-21T12:52:17.050668+0000","last_undegraded":"2024-03-21T12:52:17.050668+0000","last_fullsized":"2024-03-21T12:52:17.050668+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:30:44.704138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.163990+0000","last_change":"2024-03-21T12:51:40.090670+0000","last_active":"2024-03-21T12:52:17.163990+0000","last_peered":"2024-03-21T12:52:17.163990+0000","last_clean":"2024-03-21T12:52:17.163990+0000","last_became_active":"2024-03-21T12:51:40.090376+0000","last_became_peered":"2024-03-21T12:51:40.090376+0000","last_unstale":"2024-03-21T12:52:17.163990+0000","last_undegraded":"2024-03-21T12:52:17.163990+0000","last_fullsized":"2024-03-21T12:52:17.163990+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:35:48.022163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157641+0000","last_change":"2024-03-21T12:51:36.071020+0000","last_active":"2024-03-21T12:52:17.157641+0000","last_peered":"2024-03-21T12:52:17.157641+0000","last_clean":"2024-03-21T12:52:17.157641+0000","last_became_active":"2024-03-21T12:51:36.069867+0000","last_became_peered":"2024-03-21T12:51:36.069867+0000","last_unstale":"2024-03-21T12:52:17.157641+0000","last_undegraded":"2024-03-21T12:52:17.157641+0000","last_fullsized":"2024-03-21T12:52:17.157641+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:22:53.953657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"69'11","reported_seq":48,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.050644+0000","last_change":"2024-03-21T12:51:38.218405+0000","last_active":"2024-03-21T12:52:17.050644+0000","last_peered":"2024-03-21T12:52:17.050644+0000","last_clean":"2024-03-21T12:52:17.050644+0000","last_became_active":"2024-03-21T12:51:38.217937+0000","last_became_peered":"2024-03-21T12:51:38.217937+0000","last_unstale":"2024-03-21T12:52:17.050644+0000","last_undegraded":"2024-03-21T12:52:17.050644+0000","last_fullsized":"2024-03-21T12:52:17.050644+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:33:11.198452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179437+0000","last_change":"2024-03-21T12:51:42.096995+0000","last_active":"2024-03-21T12:52:17.179437+0000","last_peered":"2024-03-21T12:52:17.179437+0000","last_clean":"2024-03-21T12:52:17.179437+0000","last_became_active":"2024-03-21T12:51:42.096458+0000","last_became_peered":"2024-03-21T12:51:42.096458+0000","last_unstale":"2024-03-21T12:52:17.179437+0000","last_undegraded":"2024-03-21T12:52:17.179437+0000","last_fullsized":"2024-03-21T12:52:17.179437+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:50:31.751024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.050956+0000","last_change":"2024-03-21T12:51:40.093003+0000","last_active":"2024-03-21T12:52:17.050956+0000","last_peered":"2024-03-21T12:52:17.050956+0000","last_clean":"2024-03-21T12:52:17.050956+0000","last_became_active":"2024-03-21T12:51:40.092698+0000","last_became_peered":"2024-03-21T12:51:40.092698+0000","last_unstale":"2024-03-21T12:52:17.050956+0000","last_undegraded":"2024-03-21T12:52:17.050956+0000","last_fullsized":"2024-03-21T12:52:17.050956+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:09:45.273067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.163904+0000","last_change":"2024-03-21T12:51:36.077229+0000","last_active":"2024-03-21T12:52:17.163904+0000","last_peered":"2024-03-21T12:52:17.163904+0000","last_clean":"2024-03-21T12:52:17.163904+0000","last_became_active":"2024-03-21T12:51:36.076932+0000","last_became_peered":"2024-03-21T12:51:36.076932+0000","last_unstale":"2024-03-21T12:52:17.163904+0000","last_undegraded":"2024-03-21T12:52:17.163904+0000","last_fullsized":"2024-03-21T12:52:17.163904+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:43:20.705557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"69'9","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179889+0000","last_change":"2024-03-21T12:51:38.091052+0000","last_active":"2024-03-21T12:52:17.179889+0000","last_peered":"2024-03-21T12:52:17.179889+0000","last_clean":"2024-03-21T12:52:17.179889+0000","last_became_active":"2024-03-21T12:51:38.090694+0000","last_became_peered":"2024-03-21T12:51:38.090694+0000","last_unstale":"2024-03-21T12:52:17.179889+0000","last_undegraded":"2024-03-21T12:52:17.179889+0000","last_fullsized":"2024-03-21T12:52:17.179889+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:03:06.920096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.050300+0000","last_change":"2024-03-21T12:51:42.098826+0000","last_active":"2024-03-21T12:52:17.050300+0000","last_peered":"2024-03-21T12:52:17.050300+0000","last_clean":"2024-03-21T12:52:17.050300+0000","last_became_active":"2024-03-21T12:51:42.098535+0000","last_became_peered":"2024-03-21T12:51:42.098535+0000","last_unstale":"2024-03-21T12:52:17.050300+0000","last_undegraded":"2024-03-21T12:52:17.050300+0000","last_fullsized":"2024-03-21T12:52:17.050300+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:56:41.581909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179840+0000","last_change":"2024-03-21T12:51:40.090680+0000","last_active":"2024-03-21T12:52:17.179840+0000","last_peered":"2024-03-21T12:52:17.179840+0000","last_clean":"2024-03-21T12:52:17.179840+0000","last_became_active":"2024-03-21T12:51:40.090348+0000","last_became_peered":"2024-03-21T12:51:40.090348+0000","last_unstale":"2024-03-21T12:52:17.179840+0000","last_undegraded":"2024-03-21T12:52:17.179840+0000","last_fullsized":"2024-03-21T12:52:17.179840+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:10:21.164408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.052572+0000","last_change":"2024-03-21T12:51:36.076452+0000","last_active":"2024-03-21T12:52:17.052572+0000","last_peered":"2024-03-21T12:52:17.052572+0000","last_clean":"2024-03-21T12:52:17.052572+0000","last_became_active":"2024-03-21T12:51:36.076025+0000","last_became_peered":"2024-03-21T12:51:36.076025+0000","last_unstale":"2024-03-21T12:52:17.052572+0000","last_undegraded":"2024-03-21T12:52:17.052572+0000","last_fullsized":"2024-03-21T12:52:17.052572+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:38:34.645681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"69'11","reported_seq":48,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.052403+0000","last_change":"2024-03-21T12:51:38.088963+0000","last_active":"2024-03-21T12:52:17.052403+0000","last_peered":"2024-03-21T12:52:17.052403+0000","last_clean":"2024-03-21T12:52:17.052403+0000","last_became_active":"2024-03-21T12:51:38.088485+0000","last_became_peered":"2024-03-21T12:51:38.088485+0000","last_unstale":"2024-03-21T12:52:17.052403+0000","last_undegraded":"2024-03-21T12:52:17.052403+0000","last_fullsized":"2024-03-21T12:52:17.052403+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:12:17.857435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180278+0000","last_change":"2024-03-21T12:51:42.098840+0000","last_active":"2024-03-21T12:52:17.180278+0000","last_peered":"2024-03-21T12:52:17.180278+0000","last_clean":"2024-03-21T12:52:17.180278+0000","last_became_active":"2024-03-21T12:51:42.098508+0000","last_became_peered":"2024-03-21T12:51:42.098508+0000","last_unstale":"2024-03-21T12:52:17.180278+0000","last_undegraded":"2024-03-21T12:52:17.180278+0000","last_fullsized":"2024-03-21T12:52:17.180278+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:08:23.656888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"69'8","reported_seq":49,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:46.131388+0000","last_change":"2024-03-21T12:51:40.094253+0000","last_active":"2024-03-21T12:52:46.131388+0000","last_peered":"2024-03-21T12:52:46.131388+0000","last_clean":"2024-03-21T12:52:46.131388+0000","last_became_active":"2024-03-21T12:51:40.093585+0000","last_became_peered":"2024-03-21T12:51:40.093585+0000","last_unstale":"2024-03-21T12:52:46.131388+0000","last_undegraded":"2024-03-21T12:52:46.131388+0000","last_fullsized":"2024-03-21T12:52:46.131388+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:08:38.727761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.163687+0000","last_change":"2024-03-21T12:51:36.079003+0000","last_active":"2024-03-21T12:52:17.163687+0000","last_peered":"2024-03-21T12:52:17.163687+0000","last_clean":"2024-03-21T12:52:17.163687+0000","last_became_active":"2024-03-21T12:51:36.078625+0000","last_became_peered":"2024-03-21T12:51:36.078625+0000","last_unstale":"2024-03-21T12:52:17.163687+0000","last_undegraded":"2024-03-21T12:52:17.163687+0000","last_fullsized":"2024-03-21T12:52:17.163687+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:14:22.361186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"69'10","reported_seq":44,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.050478+0000","last_change":"2024-03-21T12:51:38.217895+0000","last_active":"2024-03-21T12:52:17.050478+0000","last_peered":"2024-03-21T12:52:17.050478+0000","last_clean":"2024-03-21T12:52:17.050478+0000","last_became_active":"2024-03-21T12:51:38.217176+0000","last_became_peered":"2024-03-21T12:51:38.217176+0000","last_unstale":"2024-03-21T12:52:17.050478+0000","last_undegraded":"2024-03-21T12:52:17.050478+0000","last_fullsized":"2024-03-21T12:52:17.050478+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:42:20.714369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.053100+0000","last_change":"2024-03-21T12:51:42.093000+0000","last_active":"2024-03-21T12:52:17.053100+0000","last_peered":"2024-03-21T12:52:17.053100+0000","last_clean":"2024-03-21T12:52:17.053100+0000","last_became_active":"2024-03-21T12:51:42.092582+0000","last_became_peered":"2024-03-21T12:51:42.092582+0000","last_unstale":"2024-03-21T12:52:17.053100+0000","last_undegraded":"2024-03-21T12:52:17.053100+0000","last_fullsized":"2024-03-21T12:52:17.053100+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:51:06.548012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"69'8","reported_seq":49,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:46.131103+0000","last_change":"2024-03-21T12:51:40.094130+0000","last_active":"2024-03-21T12:52:46.131103+0000","last_peered":"2024-03-21T12:52:46.131103+0000","last_clean":"2024-03-21T12:52:46.131103+0000","last_became_active":"2024-03-21T12:51:40.093765+0000","last_became_peered":"2024-03-21T12:51:40.093765+0000","last_unstale":"2024-03-21T12:52:46.131103+0000","last_undegraded":"2024-03-21T12:52:46.131103+0000","last_fullsized":"2024-03-21T12:52:46.131103+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:20:42.520650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179910+0000","last_change":"2024-03-21T12:51:36.077294+0000","last_active":"2024-03-21T12:52:17.179910+0000","last_peered":"2024-03-21T12:52:17.179910+0000","last_clean":"2024-03-21T12:52:17.179910+0000","last_became_active":"2024-03-21T12:51:36.076905+0000","last_became_peered":"2024-03-21T12:51:36.076905+0000","last_unstale":"2024-03-21T12:52:17.179910+0000","last_undegraded":"2024-03-21T12:52:17.179910+0000","last_fullsized":"2024-03-21T12:52:17.179910+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:54:29.026133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"69'9","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157806+0000","last_change":"2024-03-21T12:51:38.218797+0000","last_active":"2024-03-21T12:52:17.157806+0000","last_peered":"2024-03-21T12:52:17.157806+0000","last_clean":"2024-03-21T12:52:17.157806+0000","last_became_active":"2024-03-21T12:51:38.218300+0000","last_became_peered":"2024-03-21T12:51:38.218300+0000","last_unstale":"2024-03-21T12:52:17.157806+0000","last_undegraded":"2024-03-21T12:52:17.157806+0000","last_fullsized":"2024-03-21T12:52:17.157806+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:30:19.063311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.050403+0000","last_change":"2024-03-21T12:51:42.090614+0000","last_active":"2024-03-21T12:52:17.050403+0000","last_peered":"2024-03-21T12:52:17.050403+0000","last_clean":"2024-03-21T12:52:17.050403+0000","last_became_active":"2024-03-21T12:51:42.090264+0000","last_became_peered":"2024-03-21T12:51:42.090264+0000","last_unstale":"2024-03-21T12:52:17.050403+0000","last_undegraded":"2024-03-21T12:52:17.050403+0000","last_fullsized":"2024-03-21T12:52:17.050403+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:39:15.150234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051618+0000","last_change":"2024-03-21T12:51:40.088526+0000","last_active":"2024-03-21T12:52:17.051618+0000","last_peered":"2024-03-21T12:52:17.051618+0000","last_clean":"2024-03-21T12:52:17.051618+0000","last_became_active":"2024-03-21T12:51:40.087842+0000","last_became_peered":"2024-03-21T12:51:40.087842+0000","last_unstale":"2024-03-21T12:52:17.051618+0000","last_undegraded":"2024-03-21T12:52:17.051618+0000","last_fullsized":"2024-03-21T12:52:17.051618+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:58:14.921052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.163479+0000","last_change":"2024-03-21T12:51:36.073263+0000","last_active":"2024-03-21T12:52:17.163479+0000","last_peered":"2024-03-21T12:52:17.163479+0000","last_clean":"2024-03-21T12:52:17.163479+0000","last_became_active":"2024-03-21T12:51:36.072946+0000","last_became_peered":"2024-03-21T12:51:36.072946+0000","last_unstale":"2024-03-21T12:52:17.163479+0000","last_undegraded":"2024-03-21T12:52:17.163479+0000","last_fullsized":"2024-03-21T12:52:17.163479+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:55:08.708449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"69'9","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180781+0000","last_change":"2024-03-21T12:51:38.219245+0000","last_active":"2024-03-21T12:52:17.180781+0000","last_peered":"2024-03-21T12:52:17.180781+0000","last_clean":"2024-03-21T12:52:17.180781+0000","last_became_active":"2024-03-21T12:51:38.218894+0000","last_became_peered":"2024-03-21T12:51:38.218894+0000","last_unstale":"2024-03-21T12:52:17.180781+0000","last_undegraded":"2024-03-21T12:52:17.180781+0000","last_fullsized":"2024-03-21T12:52:17.180781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:03:21.235285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.163723+0000","last_change":"2024-03-21T12:51:42.181552+0000","last_active":"2024-03-21T12:52:17.163723+0000","last_peered":"2024-03-21T12:52:17.163723+0000","last_clean":"2024-03-21T12:52:17.163723+0000","last_became_active":"2024-03-21T12:51:42.181261+0000","last_became_peered":"2024-03-21T12:51:42.181261+0000","last_unstale":"2024-03-21T12:52:17.163723+0000","last_undegraded":"2024-03-21T12:52:17.163723+0000","last_fullsized":"2024-03-21T12:52:17.163723+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:26:33.326732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157703+0000","last_change":"2024-03-21T12:51:40.092939+0000","last_active":"2024-03-21T12:52:17.157703+0000","last_peered":"2024-03-21T12:52:17.157703+0000","last_clean":"2024-03-21T12:52:17.157703+0000","last_became_active":"2024-03-21T12:51:40.092507+0000","last_became_peered":"2024-03-21T12:51:40.092507+0000","last_unstale":"2024-03-21T12:52:17.157703+0000","last_undegraded":"2024-03-21T12:52:17.157703+0000","last_fullsized":"2024-03-21T12:52:17.157703+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:14:27.750397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"62'1","reported_seq":39,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179887+0000","last_change":"2024-03-21T12:51:36.072251+0000","last_active":"2024-03-21T12:52:17.179887+0000","last_peered":"2024-03-21T12:52:17.179887+0000","last_clean":"2024-03-21T12:52:17.179887+0000","last_became_active":"2024-03-21T12:51:36.071958+0000","last_became_peered":"2024-03-21T12:51:36.071958+0000","last_unstale":"2024-03-21T12:52:17.179887+0000","last_undegraded":"2024-03-21T12:52:17.179887+0000","last_fullsized":"2024-03-21T12:52:17.179887+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:23:27.957768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":911,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"69'6","reported_seq":38,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051802+0000","last_change":"2024-03-21T12:51:38.217923+0000","last_active":"2024-03-21T12:52:17.051802+0000","last_peered":"2024-03-21T12:52:17.051802+0000","last_clean":"2024-03-21T12:52:17.051802+0000","last_became_active":"2024-03-21T12:51:38.217169+0000","last_became_peered":"2024-03-21T12:51:38.217169+0000","last_unstale":"2024-03-21T12:52:17.051802+0000","last_undegraded":"2024-03-21T12:52:17.051802+0000","last_fullsized":"2024-03-21T12:52:17.051802+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:29:57.972260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"69'1","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157296+0000","last_change":"2024-03-21T12:51:42.180920+0000","last_active":"2024-03-21T12:52:17.157296+0000","last_peered":"2024-03-21T12:52:17.157296+0000","last_clean":"2024-03-21T12:52:17.157296+0000","last_became_active":"2024-03-21T12:51:42.180578+0000","last_became_peered":"2024-03-21T12:51:42.180578+0000","last_unstale":"2024-03-21T12:52:17.157296+0000","last_undegraded":"2024-03-21T12:52:17.157296+0000","last_fullsized":"2024-03-21T12:52:17.157296+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:48:58.686198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"62'2","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.163975+0000","last_change":"2024-03-21T12:51:36.077977+0000","last_active":"2024-03-21T12:52:17.163975+0000","last_peered":"2024-03-21T12:52:17.163975+0000","last_clean":"2024-03-21T12:52:17.163975+0000","last_became_active":"2024-03-21T12:51:36.077661+0000","last_became_peered":"2024-03-21T12:51:36.077661+0000","last_unstale":"2024-03-21T12:52:17.163975+0000","last_undegraded":"2024-03-21T12:52:17.163975+0000","last_fullsized":"2024-03-21T12:52:17.163975+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:28:13.334015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"69'15","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.156997+0000","last_change":"2024-03-21T12:51:38.219468+0000","last_active":"2024-03-21T12:52:17.156997+0000","last_peered":"2024-03-21T12:52:17.156997+0000","last_clean":"2024-03-21T12:52:17.156997+0000","last_became_active":"2024-03-21T12:51:38.219125+0000","last_became_peered":"2024-03-21T12:51:38.219125+0000","last_unstale":"2024-03-21T12:52:17.156997+0000","last_undegraded":"2024-03-21T12:52:17.156997+0000","last_fullsized":"2024-03-21T12:52:17.156997+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:49:43.933463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"69'8","reported_seq":49,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:46.131389+0000","last_change":"2024-03-21T12:51:40.090707+0000","last_active":"2024-03-21T12:52:46.131389+0000","last_peered":"2024-03-21T12:52:46.131389+0000","last_clean":"2024-03-21T12:52:46.131389+0000","last_became_active":"2024-03-21T12:51:40.090408+0000","last_became_peered":"2024-03-21T12:51:40.090408+0000","last_unstale":"2024-03-21T12:52:46.131389+0000","last_undegraded":"2024-03-21T12:52:46.131389+0000","last_fullsized":"2024-03-21T12:52:46.131389+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:38:02.647098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.050774+0000","last_change":"2024-03-21T12:51:42.095706+0000","last_active":"2024-03-21T12:52:17.050774+0000","last_peered":"2024-03-21T12:52:17.050774+0000","last_clean":"2024-03-21T12:52:17.050774+0000","last_became_active":"2024-03-21T12:51:42.094206+0000","last_became_peered":"2024-03-21T12:51:42.094206+0000","last_unstale":"2024-03-21T12:52:17.050774+0000","last_undegraded":"2024-03-21T12:52:17.050774+0000","last_fullsized":"2024-03-21T12:52:17.050774+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:09:35.491438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.164011+0000","last_change":"2024-03-21T12:51:36.070333+0000","last_active":"2024-03-21T12:52:17.164011+0000","last_peered":"2024-03-21T12:52:17.164011+0000","last_clean":"2024-03-21T12:52:17.164011+0000","last_became_active":"2024-03-21T12:51:36.070042+0000","last_became_peered":"2024-03-21T12:51:36.070042+0000","last_unstale":"2024-03-21T12:52:17.164011+0000","last_undegraded":"2024-03-21T12:52:17.164011+0000","last_fullsized":"2024-03-21T12:52:17.164011+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:58:39.629607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"69'12","reported_seq":70,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.052158+0000","last_change":"2024-03-21T12:51:52.998558+0000","last_active":"2024-03-21T12:52:17.052158+0000","last_peered":"2024-03-21T12:52:17.052158+0000","last_clean":"2024-03-21T12:52:17.052158+0000","last_became_active":"2024-03-21T12:51:52.991832+0000","last_became_peered":"2024-03-21T12:51:52.991832+0000","last_unstale":"2024-03-21T12:52:17.052158+0000","last_undegraded":"2024-03-21T12:52:17.052158+0000","last_fullsized":"2024-03-21T12:52:17.052158+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:41:43.577719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.048207+0000","last_change":"2024-03-21T12:51:40.085820+0000","last_active":"2024-03-21T12:52:17.048207+0000","last_peered":"2024-03-21T12:52:17.048207+0000","last_clean":"2024-03-21T12:52:17.048207+0000","last_became_active":"2024-03-21T12:51:40.085537+0000","last_became_peered":"2024-03-21T12:51:40.085537+0000","last_unstale":"2024-03-21T12:52:17.048207+0000","last_undegraded":"2024-03-21T12:52:17.048207+0000","last_fullsized":"2024-03-21T12:52:17.048207+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:30:01.662575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.163215+0000","last_change":"2024-03-21T12:51:42.091977+0000","last_active":"2024-03-21T12:52:17.163215+0000","last_peered":"2024-03-21T12:52:17.163215+0000","last_clean":"2024-03-21T12:52:17.163215+0000","last_became_active":"2024-03-21T12:51:42.091290+0000","last_became_peered":"2024-03-21T12:51:42.091290+0000","last_unstale":"2024-03-21T12:52:17.163215+0000","last_undegraded":"2024-03-21T12:52:17.163215+0000","last_fullsized":"2024-03-21T12:52:17.163215+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:11:31.592650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.048417+0000","last_change":"2024-03-21T12:51:40.091513+0000","last_active":"2024-03-21T12:52:17.048417+0000","last_peered":"2024-03-21T12:52:17.048417+0000","last_clean":"2024-03-21T12:52:17.048417+0000","last_became_active":"2024-03-21T12:51:40.091151+0000","last_became_peered":"2024-03-21T12:51:40.091151+0000","last_unstale":"2024-03-21T12:52:17.048417+0000","last_undegraded":"2024-03-21T12:52:17.048417+0000","last_fullsized":"2024-03-21T12:52:17.048417+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:46:16.217484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.164162+0000","last_change":"2024-03-21T12:51:36.074462+0000","last_active":"2024-03-21T12:52:17.164162+0000","last_peered":"2024-03-21T12:52:17.164162+0000","last_clean":"2024-03-21T12:52:17.164162+0000","last_became_active":"2024-03-21T12:51:36.074122+0000","last_became_peered":"2024-03-21T12:51:36.074122+0000","last_unstale":"2024-03-21T12:52:17.164162+0000","last_undegraded":"2024-03-21T12:52:17.164162+0000","last_fullsized":"2024-03-21T12:52:17.164162+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:14:16.354037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"69'19","reported_seq":60,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180184+0000","last_change":"2024-03-21T12:51:38.218339+0000","last_active":"2024-03-21T12:52:17.180184+0000","last_peered":"2024-03-21T12:52:17.180184+0000","last_clean":"2024-03-21T12:52:17.180184+0000","last_became_active":"2024-03-21T12:51:38.217932+0000","last_became_peered":"2024-03-21T12:51:38.217932+0000","last_unstale":"2024-03-21T12:52:17.180184+0000","last_undegraded":"2024-03-21T12:52:17.180184+0000","last_fullsized":"2024-03-21T12:52:17.180184+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:18:42.725674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179536+0000","last_change":"2024-03-21T12:51:42.095439+0000","last_active":"2024-03-21T12:52:17.179536+0000","last_peered":"2024-03-21T12:52:17.179536+0000","last_clean":"2024-03-21T12:52:17.179536+0000","last_became_active":"2024-03-21T12:51:42.095096+0000","last_became_peered":"2024-03-21T12:51:42.095096+0000","last_unstale":"2024-03-21T12:52:17.179536+0000","last_undegraded":"2024-03-21T12:52:17.179536+0000","last_fullsized":"2024-03-21T12:52:17.179536+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:20:53.350200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.049694+0000","last_change":"2024-03-21T12:51:40.088305+0000","last_active":"2024-03-21T12:52:17.049694+0000","last_peered":"2024-03-21T12:52:17.049694+0000","last_clean":"2024-03-21T12:52:17.049694+0000","last_became_active":"2024-03-21T12:51:40.087907+0000","last_became_peered":"2024-03-21T12:51:40.087907+0000","last_unstale":"2024-03-21T12:52:17.049694+0000","last_undegraded":"2024-03-21T12:52:17.049694+0000","last_fullsized":"2024-03-21T12:52:17.049694+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:40.168333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157344+0000","last_change":"2024-03-21T12:51:36.069544+0000","last_active":"2024-03-21T12:52:17.157344+0000","last_peered":"2024-03-21T12:52:17.157344+0000","last_clean":"2024-03-21T12:52:17.157344+0000","last_became_active":"2024-03-21T12:51:36.069231+0000","last_became_peered":"2024-03-21T12:51:36.069231+0000","last_unstale":"2024-03-21T12:52:17.157344+0000","last_undegraded":"2024-03-21T12:52:17.157344+0000","last_fullsized":"2024-03-21T12:52:17.157344+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:46:57.430679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"69'9","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.181386+0000","last_change":"2024-03-21T12:51:38.085970+0000","last_active":"2024-03-21T12:52:17.181386+0000","last_peered":"2024-03-21T12:52:17.181386+0000","last_clean":"2024-03-21T12:52:17.181386+0000","last_became_active":"2024-03-21T12:51:38.085526+0000","last_became_peered":"2024-03-21T12:51:38.085526+0000","last_unstale":"2024-03-21T12:52:17.181386+0000","last_undegraded":"2024-03-21T12:52:17.181386+0000","last_fullsized":"2024-03-21T12:52:17.181386+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:02:33.907384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051103+0000","last_change":"2024-03-21T12:51:42.101403+0000","last_active":"2024-03-21T12:52:17.051103+0000","last_peered":"2024-03-21T12:52:17.051103+0000","last_clean":"2024-03-21T12:52:17.051103+0000","last_became_active":"2024-03-21T12:51:42.100887+0000","last_became_peered":"2024-03-21T12:51:42.100887+0000","last_unstale":"2024-03-21T12:52:17.051103+0000","last_undegraded":"2024-03-21T12:52:17.051103+0000","last_fullsized":"2024-03-21T12:52:17.051103+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:35:52.065907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"65'2","reported_seq":44,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157018+0000","last_change":"2024-03-21T12:51:39.077942+0000","last_active":"2024-03-21T12:52:17.157018+0000","last_peered":"2024-03-21T12:52:17.157018+0000","last_clean":"2024-03-21T12:52:17.157018+0000","last_became_active":"2024-03-21T12:51:35.065288+0000","last_became_peered":"2024-03-21T12:51:35.065288+0000","last_unstale":"2024-03-21T12:52:17.157018+0000","last_undegraded":"2024-03-21T12:52:17.157018+0000","last_fullsized":"2024-03-21T12:52:17.157018+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:34.048471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:34.048471+0000","last_clean_scrub_stamp":"2024-03-21T12:51:34.048471+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:38:07.675192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001525169,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051957+0000","last_change":"2024-03-21T12:51:36.075601+0000","last_active":"2024-03-21T12:52:17.051957+0000","last_peered":"2024-03-21T12:52:17.051957+0000","last_clean":"2024-03-21T12:52:17.051957+0000","last_became_active":"2024-03-21T12:51:36.075231+0000","last_became_peered":"2024-03-21T12:51:36.075231+0000","last_unstale":"2024-03-21T12:52:17.051957+0000","last_undegraded":"2024-03-21T12:52:17.051957+0000","last_fullsized":"2024-03-21T12:52:17.051957+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:22:34.166997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"69'28","reported_seq":93,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:46.131036+0000","last_change":"2024-03-21T12:51:38.089283+0000","last_active":"2024-03-21T12:52:46.131036+0000","last_peered":"2024-03-21T12:52:46.131036+0000","last_clean":"2024-03-21T12:52:46.131036+0000","last_became_active":"2024-03-21T12:51:38.088682+0000","last_became_peered":"2024-03-21T12:51:38.088682+0000","last_unstale":"2024-03-21T12:52:46.131036+0000","last_undegraded":"2024-03-21T12:52:46.131036+0000","last_fullsized":"2024-03-21T12:52:46.131036+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:45:22.774914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180094+0000","last_change":"2024-03-21T12:51:40.088236+0000","last_active":"2024-03-21T12:52:17.180094+0000","last_peered":"2024-03-21T12:52:17.180094+0000","last_clean":"2024-03-21T12:52:17.180094+0000","last_became_active":"2024-03-21T12:51:40.086146+0000","last_became_peered":"2024-03-21T12:51:40.086146+0000","last_unstale":"2024-03-21T12:52:17.180094+0000","last_undegraded":"2024-03-21T12:52:17.180094+0000","last_fullsized":"2024-03-21T12:52:17.180094+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:29:43.587618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.164261+0000","last_change":"2024-03-21T12:51:42.180753+0000","last_active":"2024-03-21T12:52:17.164261+0000","last_peered":"2024-03-21T12:52:17.164261+0000","last_clean":"2024-03-21T12:52:17.164261+0000","last_became_active":"2024-03-21T12:51:42.180443+0000","last_became_peered":"2024-03-21T12:51:42.180443+0000","last_unstale":"2024-03-21T12:52:17.164261+0000","last_undegraded":"2024-03-21T12:52:17.164261+0000","last_fullsized":"2024-03-21T12:52:17.164261+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:36:29.737201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179567+0000","last_change":"2024-03-21T12:51:36.073957+0000","last_active":"2024-03-21T12:52:17.179567+0000","last_peered":"2024-03-21T12:52:17.179567+0000","last_clean":"2024-03-21T12:52:17.179567+0000","last_became_active":"2024-03-21T12:51:36.073553+0000","last_became_peered":"2024-03-21T12:51:36.073553+0000","last_unstale":"2024-03-21T12:52:17.179567+0000","last_undegraded":"2024-03-21T12:52:17.179567+0000","last_fullsized":"2024-03-21T12:52:17.179567+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:28:43.225482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.049179+0000","last_change":"2024-03-21T12:51:39.074088+0000","last_active":"2024-03-21T12:52:17.049179+0000","last_peered":"2024-03-21T12:52:17.049179+0000","last_clean":"2024-03-21T12:52:17.049179+0000","last_became_active":"2024-03-21T12:51:35.064091+0000","last_became_peered":"2024-03-21T12:51:35.064091+0000","last_unstale":"2024-03-21T12:52:17.049179+0000","last_undegraded":"2024-03-21T12:52:17.049179+0000","last_fullsized":"2024-03-21T12:52:17.049179+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:34.048471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:34.048471+0000","last_clean_scrub_stamp":"2024-03-21T12:51:34.048471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:04:28.184801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000457348,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"69'13","reported_seq":56,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180910+0000","last_change":"2024-03-21T12:51:38.090405+0000","last_active":"2024-03-21T12:52:17.180910+0000","last_peered":"2024-03-21T12:52:17.180910+0000","last_clean":"2024-03-21T12:52:17.180910+0000","last_became_active":"2024-03-21T12:51:38.090094+0000","last_became_peered":"2024-03-21T12:51:38.090094+0000","last_unstale":"2024-03-21T12:52:17.180910+0000","last_undegraded":"2024-03-21T12:52:17.180910+0000","last_fullsized":"2024-03-21T12:52:17.180910+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:56:23.536335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.048839+0000","last_change":"2024-03-21T12:51:40.094943+0000","last_active":"2024-03-21T12:52:17.048839+0000","last_peered":"2024-03-21T12:52:17.048839+0000","last_clean":"2024-03-21T12:52:17.048839+0000","last_became_active":"2024-03-21T12:51:40.092512+0000","last_became_peered":"2024-03-21T12:51:40.092512+0000","last_unstale":"2024-03-21T12:52:17.048839+0000","last_undegraded":"2024-03-21T12:52:17.048839+0000","last_fullsized":"2024-03-21T12:52:17.048839+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:00:54.524557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157678+0000","last_change":"2024-03-21T12:51:42.097990+0000","last_active":"2024-03-21T12:52:17.157678+0000","last_peered":"2024-03-21T12:52:17.157678+0000","last_clean":"2024-03-21T12:52:17.157678+0000","last_became_active":"2024-03-21T12:51:42.097683+0000","last_became_peered":"2024-03-21T12:51:42.097683+0000","last_unstale":"2024-03-21T12:52:17.157678+0000","last_undegraded":"2024-03-21T12:52:17.157678+0000","last_fullsized":"2024-03-21T12:52:17.157678+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:50:43.529755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":278,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:19.149615+0000","last_change":"2024-03-21T12:51:24.555056+0000","last_active":"2024-03-21T12:52:19.149615+0000","last_peered":"2024-03-21T12:52:19.149615+0000","last_clean":"2024-03-21T12:52:19.149615+0000","last_became_active":"2024-03-21T12:51:24.554651+0000","last_became_peered":"2024-03-21T12:51:24.554651+0000","last_unstale":"2024-03-21T12:52:19.149615+0000","last_undegraded":"2024-03-21T12:52:19.149615+0000","last_fullsized":"2024-03-21T12:52:19.149615+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:49:12.895321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:49:12.895321+0000","last_clean_scrub_stamp":"2024-03-21T12:49:12.895321+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:55:11.444449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.049670+0000","last_change":"2024-03-21T12:51:36.076902+0000","last_active":"2024-03-21T12:52:17.049670+0000","last_peered":"2024-03-21T12:52:17.049670+0000","last_clean":"2024-03-21T12:52:17.049670+0000","last_became_active":"2024-03-21T12:51:36.076579+0000","last_became_peered":"2024-03-21T12:51:36.076579+0000","last_unstale":"2024-03-21T12:52:17.049670+0000","last_undegraded":"2024-03-21T12:52:17.049670+0000","last_fullsized":"2024-03-21T12:52:17.049670+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:27:59.137686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"69'16","reported_seq":67,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:46.131583+0000","last_change":"2024-03-21T12:51:38.218501+0000","last_active":"2024-03-21T12:52:46.131583+0000","last_peered":"2024-03-21T12:52:46.131583+0000","last_clean":"2024-03-21T12:52:46.131583+0000","last_became_active":"2024-03-21T12:51:38.217927+0000","last_became_peered":"2024-03-21T12:51:38.217927+0000","last_unstale":"2024-03-21T12:52:46.131583+0000","last_undegraded":"2024-03-21T12:52:46.131583+0000","last_fullsized":"2024-03-21T12:52:46.131583+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:31:34.554432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.163819+0000","last_change":"2024-03-21T12:51:40.091597+0000","last_active":"2024-03-21T12:52:17.163819+0000","last_peered":"2024-03-21T12:52:17.163819+0000","last_clean":"2024-03-21T12:52:17.163819+0000","last_became_active":"2024-03-21T12:51:40.091293+0000","last_became_peered":"2024-03-21T12:51:40.091293+0000","last_unstale":"2024-03-21T12:52:17.163819+0000","last_undegraded":"2024-03-21T12:52:17.163819+0000","last_fullsized":"2024-03-21T12:52:17.163819+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:41:46.571271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180099+0000","last_change":"2024-03-21T12:51:42.090303+0000","last_active":"2024-03-21T12:52:17.180099+0000","last_peered":"2024-03-21T12:52:17.180099+0000","last_clean":"2024-03-21T12:52:17.180099+0000","last_became_active":"2024-03-21T12:51:42.089949+0000","last_became_peered":"2024-03-21T12:51:42.089949+0000","last_unstale":"2024-03-21T12:52:17.180099+0000","last_undegraded":"2024-03-21T12:52:17.180099+0000","last_fullsized":"2024-03-21T12:52:17.180099+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:15:49.504071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179738+0000","last_change":"2024-03-21T12:51:36.078285+0000","last_active":"2024-03-21T12:52:17.179738+0000","last_peered":"2024-03-21T12:52:17.179738+0000","last_clean":"2024-03-21T12:52:17.179738+0000","last_became_active":"2024-03-21T12:51:36.077980+0000","last_became_peered":"2024-03-21T12:51:36.077980+0000","last_unstale":"2024-03-21T12:52:17.179738+0000","last_undegraded":"2024-03-21T12:52:17.179738+0000","last_fullsized":"2024-03-21T12:52:17.179738+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:46:03.937522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"70'5","reported_seq":116,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:51.651486+0000","last_change":"2024-03-21T12:51:39.079533+0000","last_active":"2024-03-21T12:52:51.651486+0000","last_peered":"2024-03-21T12:52:51.651486+0000","last_clean":"2024-03-21T12:52:51.651486+0000","last_became_active":"2024-03-21T12:51:35.065822+0000","last_became_peered":"2024-03-21T12:51:35.065822+0000","last_unstale":"2024-03-21T12:52:51.651486+0000","last_undegraded":"2024-03-21T12:52:51.651486+0000","last_fullsized":"2024-03-21T12:52:51.651486+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:34.048471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:34.048471+0000","last_clean_scrub_stamp":"2024-03-21T12:51:34.048471+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:35:37.899070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013453339999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":71,"num_read_kb":66,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"69'12","reported_seq":47,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180382+0000","last_change":"2024-03-21T12:51:38.084972+0000","last_active":"2024-03-21T12:52:17.180382+0000","last_peered":"2024-03-21T12:52:17.180382+0000","last_clean":"2024-03-21T12:52:17.180382+0000","last_became_active":"2024-03-21T12:51:38.084609+0000","last_became_peered":"2024-03-21T12:51:38.084609+0000","last_unstale":"2024-03-21T12:52:17.180382+0000","last_undegraded":"2024-03-21T12:52:17.180382+0000","last_fullsized":"2024-03-21T12:52:17.180382+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:22:06.342925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157907+0000","last_change":"2024-03-21T12:51:40.093166+0000","last_active":"2024-03-21T12:52:17.157907+0000","last_peered":"2024-03-21T12:52:17.157907+0000","last_clean":"2024-03-21T12:52:17.157907+0000","last_became_active":"2024-03-21T12:51:40.092876+0000","last_became_peered":"2024-03-21T12:51:40.092876+0000","last_unstale":"2024-03-21T12:52:17.157907+0000","last_undegraded":"2024-03-21T12:52:17.157907+0000","last_fullsized":"2024-03-21T12:52:17.157907+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:40:55.541743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179378+0000","last_change":"2024-03-21T12:51:42.179981+0000","last_active":"2024-03-21T12:52:17.179378+0000","last_peered":"2024-03-21T12:52:17.179378+0000","last_clean":"2024-03-21T12:52:17.179378+0000","last_became_active":"2024-03-21T12:51:42.179520+0000","last_became_peered":"2024-03-21T12:51:42.179520+0000","last_unstale":"2024-03-21T12:52:17.179378+0000","last_undegraded":"2024-03-21T12:52:17.179378+0000","last_fullsized":"2024-03-21T12:52:17.179378+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:59:35.003933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"62'1","reported_seq":39,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180208+0000","last_change":"2024-03-21T12:51:36.064904+0000","last_active":"2024-03-21T12:52:17.180208+0000","last_peered":"2024-03-21T12:52:17.180208+0000","last_clean":"2024-03-21T12:52:17.180208+0000","last_became_active":"2024-03-21T12:51:36.064600+0000","last_became_peered":"2024-03-21T12:51:36.064600+0000","last_unstale":"2024-03-21T12:52:17.180208+0000","last_undegraded":"2024-03-21T12:52:17.180208+0000","last_fullsized":"2024-03-21T12:52:17.180208+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:17:06.103623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"69'19","reported_seq":65,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180995+0000","last_change":"2024-03-21T12:51:38.217478+0000","last_active":"2024-03-21T12:52:17.180995+0000","last_peered":"2024-03-21T12:52:17.180995+0000","last_clean":"2024-03-21T12:52:17.180995+0000","last_became_active":"2024-03-21T12:51:38.217190+0000","last_became_peered":"2024-03-21T12:51:38.217190+0000","last_unstale":"2024-03-21T12:52:17.180995+0000","last_undegraded":"2024-03-21T12:52:17.180995+0000","last_fullsized":"2024-03-21T12:52:17.180995+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:04:13.088640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180273+0000","last_change":"2024-03-21T12:51:40.092003+0000","last_active":"2024-03-21T12:52:17.180273+0000","last_peered":"2024-03-21T12:52:17.180273+0000","last_clean":"2024-03-21T12:52:17.180273+0000","last_became_active":"2024-03-21T12:51:40.091738+0000","last_became_peered":"2024-03-21T12:51:40.091738+0000","last_unstale":"2024-03-21T12:52:17.180273+0000","last_undegraded":"2024-03-21T12:52:17.180273+0000","last_fullsized":"2024-03-21T12:52:17.180273+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:30:09.604700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179565+0000","last_change":"2024-03-21T12:51:42.100627+0000","last_active":"2024-03-21T12:52:17.179565+0000","last_peered":"2024-03-21T12:52:17.179565+0000","last_clean":"2024-03-21T12:52:17.179565+0000","last_became_active":"2024-03-21T12:51:42.100281+0000","last_became_peered":"2024-03-21T12:51:42.100281+0000","last_unstale":"2024-03-21T12:52:17.179565+0000","last_undegraded":"2024-03-21T12:52:17.179565+0000","last_fullsized":"2024-03-21T12:52:17.179565+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:48:24.792167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157699+0000","last_change":"2024-03-21T12:51:36.070884+0000","last_active":"2024-03-21T12:52:17.157699+0000","last_peered":"2024-03-21T12:52:17.157699+0000","last_clean":"2024-03-21T12:52:17.157699+0000","last_became_active":"2024-03-21T12:51:36.069906+0000","last_became_peered":"2024-03-21T12:51:36.069906+0000","last_unstale":"2024-03-21T12:52:17.157699+0000","last_undegraded":"2024-03-21T12:52:17.157699+0000","last_fullsized":"2024-03-21T12:52:17.157699+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:31:35.512295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"69'10","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.049389+0000","last_change":"2024-03-21T12:51:52.999055+0000","last_active":"2024-03-21T12:52:17.049389+0000","last_peered":"2024-03-21T12:52:17.049389+0000","last_clean":"2024-03-21T12:52:17.049389+0000","last_became_active":"2024-03-21T12:51:52.992240+0000","last_became_peered":"2024-03-21T12:51:52.992240+0000","last_unstale":"2024-03-21T12:52:17.049389+0000","last_undegraded":"2024-03-21T12:52:17.049389+0000","last_fullsized":"2024-03-21T12:52:17.049389+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:08:06.576697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"69'8","reported_seq":49,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:46.131056+0000","last_change":"2024-03-21T12:51:40.091815+0000","last_active":"2024-03-21T12:52:46.131056+0000","last_peered":"2024-03-21T12:52:46.131056+0000","last_clean":"2024-03-21T12:52:46.131056+0000","last_became_active":"2024-03-21T12:51:40.091473+0000","last_became_peered":"2024-03-21T12:51:40.091473+0000","last_unstale":"2024-03-21T12:52:46.131056+0000","last_undegraded":"2024-03-21T12:52:46.131056+0000","last_fullsized":"2024-03-21T12:52:46.131056+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:38:46.914405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.164060+0000","last_change":"2024-03-21T12:51:42.181425+0000","last_active":"2024-03-21T12:52:17.164060+0000","last_peered":"2024-03-21T12:52:17.164060+0000","last_clean":"2024-03-21T12:52:17.164060+0000","last_became_active":"2024-03-21T12:51:42.181095+0000","last_became_peered":"2024-03-21T12:51:42.181095+0000","last_unstale":"2024-03-21T12:52:17.164060+0000","last_undegraded":"2024-03-21T12:52:17.164060+0000","last_fullsized":"2024-03-21T12:52:17.164060+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:51:39.385233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"62'1","reported_seq":34,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180535+0000","last_change":"2024-03-21T12:51:36.074641+0000","last_active":"2024-03-21T12:52:17.180535+0000","last_peered":"2024-03-21T12:52:17.180535+0000","last_clean":"2024-03-21T12:52:17.180535+0000","last_became_active":"2024-03-21T12:51:36.073664+0000","last_became_peered":"2024-03-21T12:51:36.073664+0000","last_unstale":"2024-03-21T12:52:17.180535+0000","last_undegraded":"2024-03-21T12:52:17.180535+0000","last_fullsized":"2024-03-21T12:52:17.180535+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:35:21.566345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"69'14","reported_seq":50,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051962+0000","last_change":"2024-03-21T12:51:38.086062+0000","last_active":"2024-03-21T12:52:17.051962+0000","last_peered":"2024-03-21T12:52:17.051962+0000","last_clean":"2024-03-21T12:52:17.051962+0000","last_became_active":"2024-03-21T12:51:38.085712+0000","last_became_peered":"2024-03-21T12:51:38.085712+0000","last_unstale":"2024-03-21T12:52:17.051962+0000","last_undegraded":"2024-03-21T12:52:17.051962+0000","last_fullsized":"2024-03-21T12:52:17.051962+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:20:26.791133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"69'8","reported_seq":49,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:46.131014+0000","last_change":"2024-03-21T12:51:40.097446+0000","last_active":"2024-03-21T12:52:46.131014+0000","last_peered":"2024-03-21T12:52:46.131014+0000","last_clean":"2024-03-21T12:52:46.131014+0000","last_became_active":"2024-03-21T12:51:40.096424+0000","last_became_peered":"2024-03-21T12:51:40.096424+0000","last_unstale":"2024-03-21T12:52:46.131014+0000","last_undegraded":"2024-03-21T12:52:46.131014+0000","last_fullsized":"2024-03-21T12:52:46.131014+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:52:47.609458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051972+0000","last_change":"2024-03-21T12:51:42.087579+0000","last_active":"2024-03-21T12:52:17.051972+0000","last_peered":"2024-03-21T12:52:17.051972+0000","last_clean":"2024-03-21T12:52:17.051972+0000","last_became_active":"2024-03-21T12:51:42.087267+0000","last_became_peered":"2024-03-21T12:51:42.087267+0000","last_unstale":"2024-03-21T12:52:17.051972+0000","last_undegraded":"2024-03-21T12:52:17.051972+0000","last_fullsized":"2024-03-21T12:52:17.051972+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:09:44.484937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051520+0000","last_change":"2024-03-21T12:51:36.078175+0000","last_active":"2024-03-21T12:52:17.051520+0000","last_peered":"2024-03-21T12:52:17.051520+0000","last_clean":"2024-03-21T12:52:17.051520+0000","last_became_active":"2024-03-21T12:51:36.077875+0000","last_became_peered":"2024-03-21T12:51:36.077875+0000","last_unstale":"2024-03-21T12:52:17.051520+0000","last_undegraded":"2024-03-21T12:52:17.051520+0000","last_fullsized":"2024-03-21T12:52:17.051520+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:53:47.186230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"69'18","reported_seq":61,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.050007+0000","last_change":"2024-03-21T12:51:38.217783+0000","last_active":"2024-03-21T12:52:17.050007+0000","last_peered":"2024-03-21T12:52:17.050007+0000","last_clean":"2024-03-21T12:52:17.050007+0000","last_became_active":"2024-03-21T12:51:38.217458+0000","last_became_peered":"2024-03-21T12:51:38.217458+0000","last_unstale":"2024-03-21T12:52:17.050007+0000","last_undegraded":"2024-03-21T12:52:17.050007+0000","last_fullsized":"2024-03-21T12:52:17.050007+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:17:33.669645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051798+0000","last_change":"2024-03-21T12:51:40.089429+0000","last_active":"2024-03-21T12:52:17.051798+0000","last_peered":"2024-03-21T12:52:17.051798+0000","last_clean":"2024-03-21T12:52:17.051798+0000","last_became_active":"2024-03-21T12:51:40.089097+0000","last_became_peered":"2024-03-21T12:51:40.089097+0000","last_unstale":"2024-03-21T12:52:17.051798+0000","last_undegraded":"2024-03-21T12:52:17.051798+0000","last_fullsized":"2024-03-21T12:52:17.051798+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:22:05.183786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157415+0000","last_change":"2024-03-21T12:51:42.092175+0000","last_active":"2024-03-21T12:52:17.157415+0000","last_peered":"2024-03-21T12:52:17.157415+0000","last_clean":"2024-03-21T12:52:17.157415+0000","last_became_active":"2024-03-21T12:51:42.091870+0000","last_became_peered":"2024-03-21T12:51:42.091870+0000","last_unstale":"2024-03-21T12:52:17.157415+0000","last_undegraded":"2024-03-21T12:52:17.157415+0000","last_fullsized":"2024-03-21T12:52:17.157415+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:51:22.230367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"69'8","reported_seq":52,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:46.131397+0000","last_change":"2024-03-21T12:51:40.092283+0000","last_active":"2024-03-21T12:52:46.131397+0000","last_peered":"2024-03-21T12:52:46.131397+0000","last_clean":"2024-03-21T12:52:46.131397+0000","last_became_active":"2024-03-21T12:51:40.092014+0000","last_became_peered":"2024-03-21T12:51:40.092014+0000","last_unstale":"2024-03-21T12:52:46.131397+0000","last_undegraded":"2024-03-21T12:52:46.131397+0000","last_fullsized":"2024-03-21T12:52:46.131397+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:37:19.924340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.049919+0000","last_change":"2024-03-21T12:51:36.073121+0000","last_active":"2024-03-21T12:52:17.049919+0000","last_peered":"2024-03-21T12:52:17.049919+0000","last_clean":"2024-03-21T12:52:17.049919+0000","last_became_active":"2024-03-21T12:51:36.072814+0000","last_became_peered":"2024-03-21T12:51:36.072814+0000","last_unstale":"2024-03-21T12:52:17.049919+0000","last_undegraded":"2024-03-21T12:52:17.049919+0000","last_fullsized":"2024-03-21T12:52:17.049919+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:52:54.557450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"69'15","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180995+0000","last_change":"2024-03-21T12:51:38.091725+0000","last_active":"2024-03-21T12:52:17.180995+0000","last_peered":"2024-03-21T12:52:17.180995+0000","last_clean":"2024-03-21T12:52:17.180995+0000","last_became_active":"2024-03-21T12:51:38.091209+0000","last_became_peered":"2024-03-21T12:51:38.091209+0000","last_unstale":"2024-03-21T12:52:17.180995+0000","last_undegraded":"2024-03-21T12:52:17.180995+0000","last_fullsized":"2024-03-21T12:52:17.180995+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:28:21.802719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.050384+0000","last_change":"2024-03-21T12:51:42.179420+0000","last_active":"2024-03-21T12:52:17.050384+0000","last_peered":"2024-03-21T12:52:17.050384+0000","last_clean":"2024-03-21T12:52:17.050384+0000","last_became_active":"2024-03-21T12:51:42.179010+0000","last_became_peered":"2024-03-21T12:51:42.179010+0000","last_unstale":"2024-03-21T12:52:17.050384+0000","last_undegraded":"2024-03-21T12:52:17.050384+0000","last_fullsized":"2024-03-21T12:52:17.050384+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:18:33.459751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157893+0000","last_change":"2024-03-21T12:51:40.088111+0000","last_active":"2024-03-21T12:52:17.157893+0000","last_peered":"2024-03-21T12:52:17.157893+0000","last_clean":"2024-03-21T12:52:17.157893+0000","last_became_active":"2024-03-21T12:51:40.087818+0000","last_became_peered":"2024-03-21T12:51:40.087818+0000","last_unstale":"2024-03-21T12:52:17.157893+0000","last_undegraded":"2024-03-21T12:52:17.157893+0000","last_fullsized":"2024-03-21T12:52:17.157893+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:40:48.001709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051808+0000","last_change":"2024-03-21T12:51:36.077405+0000","last_active":"2024-03-21T12:52:17.051808+0000","last_peered":"2024-03-21T12:52:17.051808+0000","last_clean":"2024-03-21T12:52:17.051808+0000","last_became_active":"2024-03-21T12:51:36.077066+0000","last_became_peered":"2024-03-21T12:51:36.077066+0000","last_unstale":"2024-03-21T12:52:17.051808+0000","last_undegraded":"2024-03-21T12:52:17.051808+0000","last_fullsized":"2024-03-21T12:52:17.051808+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:50:48.083512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"69'11","reported_seq":48,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.054203+0000","last_change":"2024-03-21T12:51:38.088586+0000","last_active":"2024-03-21T12:52:17.054203+0000","last_peered":"2024-03-21T12:52:17.054203+0000","last_clean":"2024-03-21T12:52:17.054203+0000","last_became_active":"2024-03-21T12:51:38.087930+0000","last_became_peered":"2024-03-21T12:51:38.087930+0000","last_unstale":"2024-03-21T12:52:17.054203+0000","last_undegraded":"2024-03-21T12:52:17.054203+0000","last_fullsized":"2024-03-21T12:52:17.054203+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:25:58.539731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.049208+0000","last_change":"2024-03-21T12:51:42.095560+0000","last_active":"2024-03-21T12:52:17.049208+0000","last_peered":"2024-03-21T12:52:17.049208+0000","last_clean":"2024-03-21T12:52:17.049208+0000","last_became_active":"2024-03-21T12:51:42.092194+0000","last_became_peered":"2024-03-21T12:51:42.092194+0000","last_unstale":"2024-03-21T12:52:17.049208+0000","last_undegraded":"2024-03-21T12:52:17.049208+0000","last_fullsized":"2024-03-21T12:52:17.049208+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:18:32.865558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180374+0000","last_change":"2024-03-21T12:51:40.081988+0000","last_active":"2024-03-21T12:52:17.180374+0000","last_peered":"2024-03-21T12:52:17.180374+0000","last_clean":"2024-03-21T12:52:17.180374+0000","last_became_active":"2024-03-21T12:51:40.081670+0000","last_became_peered":"2024-03-21T12:51:40.081670+0000","last_unstale":"2024-03-21T12:52:17.180374+0000","last_undegraded":"2024-03-21T12:52:17.180374+0000","last_fullsized":"2024-03-21T12:52:17.180374+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:04:42.492985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180031+0000","last_change":"2024-03-21T12:51:36.072552+0000","last_active":"2024-03-21T12:52:17.180031+0000","last_peered":"2024-03-21T12:52:17.180031+0000","last_clean":"2024-03-21T12:52:17.180031+0000","last_became_active":"2024-03-21T12:51:36.072254+0000","last_became_peered":"2024-03-21T12:51:36.072254+0000","last_unstale":"2024-03-21T12:52:17.180031+0000","last_undegraded":"2024-03-21T12:52:17.180031+0000","last_fullsized":"2024-03-21T12:52:17.180031+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:24:18.563715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"69'17","reported_seq":57,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.053143+0000","last_change":"2024-03-21T12:51:38.088641+0000","last_active":"2024-03-21T12:52:17.053143+0000","last_peered":"2024-03-21T12:52:17.053143+0000","last_clean":"2024-03-21T12:52:17.053143+0000","last_became_active":"2024-03-21T12:51:38.088079+0000","last_became_peered":"2024-03-21T12:51:38.088079+0000","last_unstale":"2024-03-21T12:52:17.053143+0000","last_undegraded":"2024-03-21T12:52:17.053143+0000","last_fullsized":"2024-03-21T12:52:17.053143+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:35:39.049497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.053702+0000","last_change":"2024-03-21T12:51:42.092898+0000","last_active":"2024-03-21T12:52:17.053702+0000","last_peered":"2024-03-21T12:52:17.053702+0000","last_clean":"2024-03-21T12:52:17.053702+0000","last_became_active":"2024-03-21T12:51:42.092540+0000","last_became_peered":"2024-03-21T12:51:42.092540+0000","last_unstale":"2024-03-21T12:52:17.053702+0000","last_undegraded":"2024-03-21T12:52:17.053702+0000","last_fullsized":"2024-03-21T12:52:17.053702+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:09:35.569648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"69'8","reported_seq":49,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:46.131074+0000","last_change":"2024-03-21T12:51:40.093377+0000","last_active":"2024-03-21T12:52:46.131074+0000","last_peered":"2024-03-21T12:52:46.131074+0000","last_clean":"2024-03-21T12:52:46.131074+0000","last_became_active":"2024-03-21T12:51:40.093049+0000","last_became_peered":"2024-03-21T12:51:40.093049+0000","last_unstale":"2024-03-21T12:52:46.131074+0000","last_undegraded":"2024-03-21T12:52:46.131074+0000","last_fullsized":"2024-03-21T12:52:46.131074+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:53:00.969293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051042+0000","last_change":"2024-03-21T12:51:36.080420+0000","last_active":"2024-03-21T12:52:17.051042+0000","last_peered":"2024-03-21T12:52:17.051042+0000","last_clean":"2024-03-21T12:52:17.051042+0000","last_became_active":"2024-03-21T12:51:36.080089+0000","last_became_peered":"2024-03-21T12:51:36.080089+0000","last_unstale":"2024-03-21T12:52:17.051042+0000","last_undegraded":"2024-03-21T12:52:17.051042+0000","last_fullsized":"2024-03-21T12:52:17.051042+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:24:16.149768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"69'10","reported_seq":44,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.052226+0000","last_change":"2024-03-21T12:51:38.084752+0000","last_active":"2024-03-21T12:52:17.052226+0000","last_peered":"2024-03-21T12:52:17.052226+0000","last_clean":"2024-03-21T12:52:17.052226+0000","last_became_active":"2024-03-21T12:51:38.084371+0000","last_became_peered":"2024-03-21T12:51:38.084371+0000","last_unstale":"2024-03-21T12:52:17.052226+0000","last_undegraded":"2024-03-21T12:52:17.052226+0000","last_fullsized":"2024-03-21T12:52:17.052226+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:51:41.382057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":383,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1531,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":66,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201300,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28304,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716848,"statfs":{"total":95995035648,"available":95966052352,"internally_reserved":0,"allocated":1318912,"data_stored":874660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462936,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28296,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716856,"statfs":{"total":95995035648,"available":95966060544,"internally_reserved":0,"allocated":1306624,"data_stored":874532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717388,"statfs":{"total":95995035648,"available":95966605312,"internally_reserved":0,"allocated":761856,"data_stored":285131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32476,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712676,"statfs":{"total":95995035648,"available":95961780224,"internally_reserved":0,"allocated":1331200,"data_stored":875326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411374,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31876,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713276,"statfs":{"total":95995035648,"available":95962394624,"internally_reserved":0,"allocated":712704,"data_stored":284011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149737,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27744,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717408,"statfs":{"total":95995035648,"available":95966625792,"internally_reserved":0,"allocated":741376,"data_stored":284131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986212,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717416,"statfs":{"total":95995035648,"available":95966633984,"internally_reserved":0,"allocated":733184,"data_stored":283845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724575,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31888,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713264,"statfs":{"total":95995035648,"available":95962382336,"internally_reserved":0,"allocated":724992,"data_stored":285036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T12:52:55.529 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-21T12:52:55.774 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:52:55.895 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:55 smithi043 ceph-mon[30062]: from='client.14739 v1:172.21.15.43:0/1788132522' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T12:52:55.896 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:55 smithi043 ceph-mon[41699]: from='client.14739 v1:172.21.15.43:0/1788132522' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T12:52:55.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:55 smithi155 ceph-mon[33846]: from='client.14739 v1:172.21.15.43:0/1788132522' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T12:52:56.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:56 smithi043 ceph-mon[30062]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:52:56.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:56 smithi043 ceph-mon[41699]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:52:56.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:56 smithi155 ceph-mon[33846]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:52:58.298 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:52:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:52:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:52:58.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:52:58 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:52:58.867 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:58 smithi043 ceph-mon[30062]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:52:58.867 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:58 smithi043 ceph-mon[41699]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:52:58.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.865+0000 7f29678e8700 1 -- 172.21.15.43:0/108185345 >> v1:172.21.15.43:6789/0 conn(0x7f296010af80 legacy=0x7f296010d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:58.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.865+0000 7f29678e8700 1 -- 172.21.15.43:0/108185345 shutdown_connections 2024-03-21T12:52:58.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.866+0000 7f29678e8700 1 -- 172.21.15.43:0/108185345 >> 172.21.15.43:0/108185345 conn(0x7f29600fc960 msgr2=0x7f29600fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:58.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.866+0000 7f29678e8700 1 -- 172.21.15.43:0/108185345 shutdown_connections 2024-03-21T12:52:58.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.866+0000 7f29678e8700 1 -- 172.21.15.43:0/108185345 wait complete. 2024-03-21T12:52:58.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.866+0000 7f29678e8700 1 Processor -- start 2024-03-21T12:52:58.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.866+0000 7f29678e8700 1 -- start start 2024-03-21T12:52:58.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.866+0000 7f29678e8700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f29601a9210 con 0x7f296010ea90 2024-03-21T12:52:58.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.866+0000 7f29678e8700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f29601aa430 con 0x7f296010af80 2024-03-21T12:52:58.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.866+0000 7f29678e8700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f29601ab650 con 0x7f2960103170 2024-03-21T12:52:58.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.867+0000 7f2965684700 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7f2960103170 0x7f296010a720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:40868/0 (socket says 172.21.15.43:40868) 2024-03-21T12:52:58.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.867+0000 7f2965684700 1 -- 172.21.15.43:0/2049333305 learned_addr learned my addr 172.21.15.43:0/2049333305 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:52:58.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.867+0000 7f29567fc700 1 -- 172.21.15.43:0/2049333305 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3806303969 0 0) 0x7f29601a9210 con 0x7f296010ea90 2024-03-21T12:52:58.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.867+0000 7f29567fc700 1 -- 172.21.15.43:0/2049333305 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f294c0031f0 con 0x7f296010ea90 2024-03-21T12:52:58.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.867+0000 7f29567fc700 1 -- 172.21.15.43:0/2049333305 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3484846651 0 0) 0x7f29601ab650 con 0x7f2960103170 2024-03-21T12:52:58.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.867+0000 7f29567fc700 1 -- 172.21.15.43:0/2049333305 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29601a9210 con 0x7f2960103170 2024-03-21T12:52:58.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.867+0000 7f29567fc700 1 -- 172.21.15.43:0/2049333305 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2276013643 0 0) 0x7f29601aa430 con 0x7f296010af80 2024-03-21T12:52:58.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.867+0000 7f29567fc700 1 -- 172.21.15.43:0/2049333305 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29601ab650 con 0x7f296010af80 2024-03-21T12:52:58.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.867+0000 7f29567fc700 1 -- 172.21.15.43:0/2049333305 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3547983070 0 0) 0x7f294c0031f0 con 0x7f296010ea90 2024-03-21T12:52:58.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.867+0000 7f29567fc700 1 -- 172.21.15.43:0/2049333305 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f29601aa430 con 0x7f296010ea90 2024-03-21T12:52:58.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.867+0000 7f29567fc700 1 -- 172.21.15.43:0/2049333305 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f2958001590 con 0x7f296010ea90 2024-03-21T12:52:58.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.867+0000 7f29567fc700 1 -- 172.21.15.43:0/2049333305 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1914266930 0 0) 0x7f29601a9210 con 0x7f2960103170 2024-03-21T12:52:58.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.867+0000 7f29567fc700 1 -- 172.21.15.43:0/2049333305 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f294c0031f0 con 0x7f2960103170 2024-03-21T12:52:58.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.867+0000 7f29567fc700 1 -- 172.21.15.43:0/2049333305 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f295c001870 con 0x7f2960103170 2024-03-21T12:52:58.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.867+0000 7f29567fc700 1 -- 172.21.15.43:0/2049333305 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1312351722 0 0) 0x7f29601ab650 con 0x7f296010af80 2024-03-21T12:52:58.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.867+0000 7f29567fc700 1 -- 172.21.15.43:0/2049333305 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f29601a9210 con 0x7f296010af80 2024-03-21T12:52:58.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.867+0000 7f29567fc700 1 -- 172.21.15.43:0/2049333305 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f29500044f0 con 0x7f296010af80 2024-03-21T12:52:58.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.867+0000 7f29567fc700 1 -- 172.21.15.43:0/2049333305 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1570386676 0 0) 0x7f29601aa430 con 0x7f296010ea90 2024-03-21T12:52:58.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.867+0000 7f29567fc700 1 -- 172.21.15.43:0/2049333305 >> v1:172.21.15.43:6790/0 conn(0x7f2960103170 legacy=0x7f296010a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:58.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.867+0000 7f29567fc700 1 -- 172.21.15.43:0/2049333305 >> v1:172.21.15.155:6789/0 conn(0x7f296010af80 legacy=0x7f29601a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:58.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.867+0000 7f29567fc700 1 -- 172.21.15.43:0/2049333305 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29601ac890 con 0x7f296010ea90 2024-03-21T12:52:58.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.867+0000 7f29678e8700 1 -- 172.21.15.43:0/2049333305 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29601ab880 con 0x7f296010ea90 2024-03-21T12:52:58.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.867+0000 7f29678e8700 1 -- 172.21.15.43:0/2049333305 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f29601abd40 con 0x7f296010ea90 2024-03-21T12:52:58.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.868+0000 7f29567fc700 1 -- 172.21.15.43:0/2049333305 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f2958003e90 con 0x7f296010ea90 2024-03-21T12:52:58.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.868+0000 7f29567fc700 1 -- 172.21.15.43:0/2049333305 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f2958004cc0 con 0x7f296010ea90 2024-03-21T12:52:58.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.868+0000 7f29678e8700 1 -- 172.21.15.43:0/2049333305 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2944005150 con 0x7f296010ea90 2024-03-21T12:52:58.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.869+0000 7f29567fc700 1 -- 172.21.15.43:0/2049333305 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7f2958003a00 con 0x7f296010ea90 2024-03-21T12:52:58.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.870+0000 7f29567fc700 1 -- 172.21.15.43:0/2049333305 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 915400759 0 0) 0x7f29580887c0 con 0x7f296010ea90 2024-03-21T12:52:58.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:58.874+0000 7f29567fc700 1 -- 172.21.15.43:0/2049333305 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f2958055df0 con 0x7f296010ea90 2024-03-21T12:52:58.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:58 smithi155 ceph-mon[33846]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:52:59.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:59.073+0000 7f29678e8700 1 -- 172.21.15.43:0/2049333305 --> v1:172.21.15.43:6800/4199387216 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2944000bc0 con 0x7f294c06f230 2024-03-21T12:52:59.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:59.083+0000 7f29567fc700 1 -- 172.21.15.43:0/2049333305 <== mgr.34493 v1:172.21.15.43:6800/4199387216 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+346466 (unknown 2965378022 0 894207582) 0x7f2944000bc0 con 0x7f294c06f230 2024-03-21T12:52:59.088 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:52:59.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:59.086+0000 7f29678e8700 1 -- 172.21.15.43:0/2049333305 >> v1:172.21.15.43:6800/4199387216 conn(0x7f294c06f230 legacy=0x7f294c0716f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:59.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:59.086+0000 7f29678e8700 1 -- 172.21.15.43:0/2049333305 >> v1:172.21.15.43:6789/0 conn(0x7f296010ea90 legacy=0x7f29601a7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:52:59.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:59.086+0000 7f29678e8700 1 -- 172.21.15.43:0/2049333305 shutdown_connections 2024-03-21T12:52:59.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:59.086+0000 7f29678e8700 1 -- 172.21.15.43:0/2049333305 >> 172.21.15.43:0/2049333305 conn(0x7f29600fc960 msgr2=0x7f29600fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:52:59.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:59.086+0000 7f29678e8700 1 -- 172.21.15.43:0/2049333305 shutdown_connections 2024-03-21T12:52:59.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:52:59.086+0000 7f29678e8700 1 -- 172.21.15.43:0/2049333305 wait complete. 2024-03-21T12:52:59.091 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-03-21T12:52:59.818 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:52:59 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:52:59.819 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:52:59 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:52:59.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:52:59 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:53:00.648 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-03-21T12:52:59.068871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596392,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1010,"num_read_kb":856,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1771434,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":600,"ondisk_log_size":600,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":236084,"kb_used_data":7452,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":749725132,"statfs":{"total":767960285184,"available":767718535168,"internally_reserved":0,"allocated":7630848,"data_stored":4046672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":234016304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003174"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051825+0000","last_change":"2024-03-21T12:51:42.088178+0000","last_active":"2024-03-21T12:52:17.051825+0000","last_peered":"2024-03-21T12:52:17.051825+0000","last_clean":"2024-03-21T12:52:17.051825+0000","last_became_active":"2024-03-21T12:51:42.087895+0000","last_became_peered":"2024-03-21T12:51:42.087895+0000","last_unstale":"2024-03-21T12:52:17.051825+0000","last_undegraded":"2024-03-21T12:52:17.051825+0000","last_fullsized":"2024-03-21T12:52:17.051825+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:07:06.943938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179443+0000","last_change":"2024-03-21T12:51:40.091222+0000","last_active":"2024-03-21T12:52:17.179443+0000","last_peered":"2024-03-21T12:52:17.179443+0000","last_clean":"2024-03-21T12:52:17.179443+0000","last_became_active":"2024-03-21T12:51:40.089461+0000","last_became_peered":"2024-03-21T12:51:40.089461+0000","last_unstale":"2024-03-21T12:52:17.179443+0000","last_undegraded":"2024-03-21T12:52:17.179443+0000","last_fullsized":"2024-03-21T12:52:17.179443+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:21:07.377799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180175+0000","last_change":"2024-03-21T12:51:36.075348+0000","last_active":"2024-03-21T12:52:17.180175+0000","last_peered":"2024-03-21T12:52:17.180175+0000","last_clean":"2024-03-21T12:52:17.180175+0000","last_became_active":"2024-03-21T12:51:36.075007+0000","last_became_peered":"2024-03-21T12:51:36.075007+0000","last_unstale":"2024-03-21T12:52:17.180175+0000","last_undegraded":"2024-03-21T12:52:17.180175+0000","last_fullsized":"2024-03-21T12:52:17.180175+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:19:54.724289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"69'9","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051764+0000","last_change":"2024-03-21T12:51:38.088822+0000","last_active":"2024-03-21T12:52:17.051764+0000","last_peered":"2024-03-21T12:52:17.051764+0000","last_clean":"2024-03-21T12:52:17.051764+0000","last_became_active":"2024-03-21T12:51:38.088084+0000","last_became_peered":"2024-03-21T12:51:38.088084+0000","last_unstale":"2024-03-21T12:52:17.051764+0000","last_undegraded":"2024-03-21T12:52:17.051764+0000","last_fullsized":"2024-03-21T12:52:17.051764+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:13:44.640011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.050866+0000","last_change":"2024-03-21T12:51:42.181273+0000","last_active":"2024-03-21T12:52:17.050866+0000","last_peered":"2024-03-21T12:52:17.050866+0000","last_clean":"2024-03-21T12:52:17.050866+0000","last_became_active":"2024-03-21T12:51:42.180885+0000","last_became_peered":"2024-03-21T12:51:42.180885+0000","last_unstale":"2024-03-21T12:52:17.050866+0000","last_undegraded":"2024-03-21T12:52:17.050866+0000","last_fullsized":"2024-03-21T12:52:17.050866+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:35:57.826722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051771+0000","last_change":"2024-03-21T12:51:40.087635+0000","last_active":"2024-03-21T12:52:17.051771+0000","last_peered":"2024-03-21T12:52:17.051771+0000","last_clean":"2024-03-21T12:52:17.051771+0000","last_became_active":"2024-03-21T12:51:40.087342+0000","last_became_peered":"2024-03-21T12:51:40.087342+0000","last_unstale":"2024-03-21T12:52:17.051771+0000","last_undegraded":"2024-03-21T12:52:17.051771+0000","last_fullsized":"2024-03-21T12:52:17.051771+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:39:55.056529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051378+0000","last_change":"2024-03-21T12:51:36.073811+0000","last_active":"2024-03-21T12:52:17.051378+0000","last_peered":"2024-03-21T12:52:17.051378+0000","last_clean":"2024-03-21T12:52:17.051378+0000","last_became_active":"2024-03-21T12:51:36.073489+0000","last_became_peered":"2024-03-21T12:51:36.073489+0000","last_unstale":"2024-03-21T12:52:17.051378+0000","last_undegraded":"2024-03-21T12:52:17.051378+0000","last_fullsized":"2024-03-21T12:52:17.051378+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:33:48.629162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"69'15","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.050775+0000","last_change":"2024-03-21T12:51:38.081956+0000","last_active":"2024-03-21T12:52:17.050775+0000","last_peered":"2024-03-21T12:52:17.050775+0000","last_clean":"2024-03-21T12:52:17.050775+0000","last_became_active":"2024-03-21T12:51:38.081141+0000","last_became_peered":"2024-03-21T12:51:38.081141+0000","last_unstale":"2024-03-21T12:52:17.050775+0000","last_undegraded":"2024-03-21T12:52:17.050775+0000","last_fullsized":"2024-03-21T12:52:17.050775+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:27:55.935543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180339+0000","last_change":"2024-03-21T12:51:42.097783+0000","last_active":"2024-03-21T12:52:17.180339+0000","last_peered":"2024-03-21T12:52:17.180339+0000","last_clean":"2024-03-21T12:52:17.180339+0000","last_became_active":"2024-03-21T12:51:42.097376+0000","last_became_peered":"2024-03-21T12:51:42.097376+0000","last_unstale":"2024-03-21T12:52:17.180339+0000","last_undegraded":"2024-03-21T12:52:17.180339+0000","last_fullsized":"2024-03-21T12:52:17.180339+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:53:25.723317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157238+0000","last_change":"2024-03-21T12:51:40.094677+0000","last_active":"2024-03-21T12:52:17.157238+0000","last_peered":"2024-03-21T12:52:17.157238+0000","last_clean":"2024-03-21T12:52:17.157238+0000","last_became_active":"2024-03-21T12:51:40.094278+0000","last_became_peered":"2024-03-21T12:51:40.094278+0000","last_unstale":"2024-03-21T12:52:17.157238+0000","last_undegraded":"2024-03-21T12:52:17.157238+0000","last_fullsized":"2024-03-21T12:52:17.157238+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:07:38.302277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157565+0000","last_change":"2024-03-21T12:51:36.073879+0000","last_active":"2024-03-21T12:52:17.157565+0000","last_peered":"2024-03-21T12:52:17.157565+0000","last_clean":"2024-03-21T12:52:17.157565+0000","last_became_active":"2024-03-21T12:51:36.073590+0000","last_became_peered":"2024-03-21T12:51:36.073590+0000","last_unstale":"2024-03-21T12:52:17.157565+0000","last_undegraded":"2024-03-21T12:52:17.157565+0000","last_fullsized":"2024-03-21T12:52:17.157565+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:57:35.709674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"69'9","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.052376+0000","last_change":"2024-03-21T12:51:38.084906+0000","last_active":"2024-03-21T12:52:17.052376+0000","last_peered":"2024-03-21T12:52:17.052376+0000","last_clean":"2024-03-21T12:52:17.052376+0000","last_became_active":"2024-03-21T12:51:38.084472+0000","last_became_peered":"2024-03-21T12:51:38.084472+0000","last_unstale":"2024-03-21T12:52:17.052376+0000","last_undegraded":"2024-03-21T12:52:17.052376+0000","last_fullsized":"2024-03-21T12:52:17.052376+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:10:54.770995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.156919+0000","last_change":"2024-03-21T12:51:42.087206+0000","last_active":"2024-03-21T12:52:17.156919+0000","last_peered":"2024-03-21T12:52:17.156919+0000","last_clean":"2024-03-21T12:52:17.156919+0000","last_became_active":"2024-03-21T12:51:42.086771+0000","last_became_peered":"2024-03-21T12:51:42.086771+0000","last_unstale":"2024-03-21T12:52:17.156919+0000","last_undegraded":"2024-03-21T12:52:17.156919+0000","last_fullsized":"2024-03-21T12:52:17.156919+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:40:35.449173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.163378+0000","last_change":"2024-03-21T12:51:40.092051+0000","last_active":"2024-03-21T12:52:17.163378+0000","last_peered":"2024-03-21T12:52:17.163378+0000","last_clean":"2024-03-21T12:52:17.163378+0000","last_became_active":"2024-03-21T12:51:40.091783+0000","last_became_peered":"2024-03-21T12:51:40.091783+0000","last_unstale":"2024-03-21T12:52:17.163378+0000","last_undegraded":"2024-03-21T12:52:17.163378+0000","last_fullsized":"2024-03-21T12:52:17.163378+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:47:33.547754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157294+0000","last_change":"2024-03-21T12:51:36.068971+0000","last_active":"2024-03-21T12:52:17.157294+0000","last_peered":"2024-03-21T12:52:17.157294+0000","last_clean":"2024-03-21T12:52:17.157294+0000","last_became_active":"2024-03-21T12:51:36.068652+0000","last_became_peered":"2024-03-21T12:51:36.068652+0000","last_unstale":"2024-03-21T12:52:17.157294+0000","last_undegraded":"2024-03-21T12:52:17.157294+0000","last_fullsized":"2024-03-21T12:52:17.157294+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:25:21.661020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"69'5","reported_seq":39,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.052212+0000","last_change":"2024-03-21T12:51:38.088696+0000","last_active":"2024-03-21T12:52:17.052212+0000","last_peered":"2024-03-21T12:52:17.052212+0000","last_clean":"2024-03-21T12:52:17.052212+0000","last_became_active":"2024-03-21T12:51:38.088249+0000","last_became_peered":"2024-03-21T12:51:38.088249+0000","last_unstale":"2024-03-21T12:52:17.052212+0000","last_undegraded":"2024-03-21T12:52:17.052212+0000","last_fullsized":"2024-03-21T12:52:17.052212+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:21:23.040367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.053471+0000","last_change":"2024-03-21T12:51:42.179297+0000","last_active":"2024-03-21T12:52:17.053471+0000","last_peered":"2024-03-21T12:52:17.053471+0000","last_clean":"2024-03-21T12:52:17.053471+0000","last_became_active":"2024-03-21T12:51:42.178990+0000","last_became_peered":"2024-03-21T12:51:42.178990+0000","last_unstale":"2024-03-21T12:52:17.053471+0000","last_undegraded":"2024-03-21T12:52:17.053471+0000","last_fullsized":"2024-03-21T12:52:17.053471+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:49:36.111327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180469+0000","last_change":"2024-03-21T12:51:40.084280+0000","last_active":"2024-03-21T12:52:17.180469+0000","last_peered":"2024-03-21T12:52:17.180469+0000","last_clean":"2024-03-21T12:52:17.180469+0000","last_became_active":"2024-03-21T12:51:40.083940+0000","last_became_peered":"2024-03-21T12:51:40.083940+0000","last_unstale":"2024-03-21T12:52:17.180469+0000","last_undegraded":"2024-03-21T12:52:17.180469+0000","last_fullsized":"2024-03-21T12:52:17.180469+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:48:19.807821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180896+0000","last_change":"2024-03-21T12:51:36.073926+0000","last_active":"2024-03-21T12:52:17.180896+0000","last_peered":"2024-03-21T12:52:17.180896+0000","last_clean":"2024-03-21T12:52:17.180896+0000","last_became_active":"2024-03-21T12:51:36.073613+0000","last_became_peered":"2024-03-21T12:51:36.073613+0000","last_unstale":"2024-03-21T12:52:17.180896+0000","last_undegraded":"2024-03-21T12:52:17.180896+0000","last_fullsized":"2024-03-21T12:52:17.180896+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:01:12.322618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"69'15","reported_seq":76,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.049949+0000","last_change":"2024-03-21T12:51:53.006013+0000","last_active":"2024-03-21T12:52:17.049949+0000","last_peered":"2024-03-21T12:52:17.049949+0000","last_clean":"2024-03-21T12:52:17.049949+0000","last_became_active":"2024-03-21T12:51:52.993106+0000","last_became_peered":"2024-03-21T12:51:52.993106+0000","last_unstale":"2024-03-21T12:52:17.049949+0000","last_undegraded":"2024-03-21T12:52:17.049949+0000","last_fullsized":"2024-03-21T12:52:17.049949+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:56:15.803027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051713+0000","last_change":"2024-03-21T12:51:42.179543+0000","last_active":"2024-03-21T12:52:17.051713+0000","last_peered":"2024-03-21T12:52:17.051713+0000","last_clean":"2024-03-21T12:52:17.051713+0000","last_became_active":"2024-03-21T12:51:42.179167+0000","last_became_peered":"2024-03-21T12:51:42.179167+0000","last_unstale":"2024-03-21T12:52:17.051713+0000","last_undegraded":"2024-03-21T12:52:17.051713+0000","last_fullsized":"2024-03-21T12:52:17.051713+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:36:00.727087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.052210+0000","last_change":"2024-03-21T12:51:40.090061+0000","last_active":"2024-03-21T12:52:17.052210+0000","last_peered":"2024-03-21T12:52:17.052210+0000","last_clean":"2024-03-21T12:52:17.052210+0000","last_became_active":"2024-03-21T12:51:40.089710+0000","last_became_peered":"2024-03-21T12:51:40.089710+0000","last_unstale":"2024-03-21T12:52:17.052210+0000","last_undegraded":"2024-03-21T12:52:17.052210+0000","last_fullsized":"2024-03-21T12:52:17.052210+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:33:59.033849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.052425+0000","last_change":"2024-03-21T12:51:36.077115+0000","last_active":"2024-03-21T12:52:17.052425+0000","last_peered":"2024-03-21T12:52:17.052425+0000","last_clean":"2024-03-21T12:52:17.052425+0000","last_became_active":"2024-03-21T12:51:36.076799+0000","last_became_peered":"2024-03-21T12:51:36.076799+0000","last_unstale":"2024-03-21T12:52:17.052425+0000","last_undegraded":"2024-03-21T12:52:17.052425+0000","last_fullsized":"2024-03-21T12:52:17.052425+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:01:43.648779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"69'12","reported_seq":52,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051170+0000","last_change":"2024-03-21T12:51:38.084344+0000","last_active":"2024-03-21T12:52:17.051170+0000","last_peered":"2024-03-21T12:52:17.051170+0000","last_clean":"2024-03-21T12:52:17.051170+0000","last_became_active":"2024-03-21T12:51:38.084029+0000","last_became_peered":"2024-03-21T12:51:38.084029+0000","last_unstale":"2024-03-21T12:52:17.051170+0000","last_undegraded":"2024-03-21T12:52:17.051170+0000","last_fullsized":"2024-03-21T12:52:17.051170+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:55:38.851456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"69'1","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.163583+0000","last_change":"2024-03-21T12:51:42.094307+0000","last_active":"2024-03-21T12:52:17.163583+0000","last_peered":"2024-03-21T12:52:17.163583+0000","last_clean":"2024-03-21T12:52:17.163583+0000","last_became_active":"2024-03-21T12:51:42.093994+0000","last_became_peered":"2024-03-21T12:51:42.093994+0000","last_unstale":"2024-03-21T12:52:17.163583+0000","last_undegraded":"2024-03-21T12:52:17.163583+0000","last_fullsized":"2024-03-21T12:52:17.163583+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:13:57.757683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":370,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"69'8","reported_seq":53,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:51.132480+0000","last_change":"2024-03-21T12:51:40.087776+0000","last_active":"2024-03-21T12:52:51.132480+0000","last_peered":"2024-03-21T12:52:51.132480+0000","last_clean":"2024-03-21T12:52:51.132480+0000","last_became_active":"2024-03-21T12:51:40.087457+0000","last_became_peered":"2024-03-21T12:51:40.087457+0000","last_unstale":"2024-03-21T12:52:51.132480+0000","last_undegraded":"2024-03-21T12:52:51.132480+0000","last_fullsized":"2024-03-21T12:52:51.132480+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:13:56.709428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"62'1","reported_seq":34,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179514+0000","last_change":"2024-03-21T12:51:36.070247+0000","last_active":"2024-03-21T12:52:17.179514+0000","last_peered":"2024-03-21T12:52:17.179514+0000","last_clean":"2024-03-21T12:52:17.179514+0000","last_became_active":"2024-03-21T12:51:36.069956+0000","last_became_peered":"2024-03-21T12:51:36.069956+0000","last_unstale":"2024-03-21T12:52:17.179514+0000","last_undegraded":"2024-03-21T12:52:17.179514+0000","last_fullsized":"2024-03-21T12:52:17.179514+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:31:35.087019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"69'10","reported_seq":44,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180649+0000","last_change":"2024-03-21T12:51:38.217798+0000","last_active":"2024-03-21T12:52:17.180649+0000","last_peered":"2024-03-21T12:52:17.180649+0000","last_clean":"2024-03-21T12:52:17.180649+0000","last_became_active":"2024-03-21T12:51:38.217467+0000","last_became_peered":"2024-03-21T12:51:38.217467+0000","last_unstale":"2024-03-21T12:52:17.180649+0000","last_undegraded":"2024-03-21T12:52:17.180649+0000","last_fullsized":"2024-03-21T12:52:17.180649+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:13:10.608259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180625+0000","last_change":"2024-03-21T12:51:42.084785+0000","last_active":"2024-03-21T12:52:17.180625+0000","last_peered":"2024-03-21T12:52:17.180625+0000","last_clean":"2024-03-21T12:52:17.180625+0000","last_became_active":"2024-03-21T12:51:42.084320+0000","last_became_peered":"2024-03-21T12:51:42.084320+0000","last_unstale":"2024-03-21T12:52:17.180625+0000","last_undegraded":"2024-03-21T12:52:17.180625+0000","last_fullsized":"2024-03-21T12:52:17.180625+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:45:32.051347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180977+0000","last_change":"2024-03-21T12:51:40.087946+0000","last_active":"2024-03-21T12:52:17.180977+0000","last_peered":"2024-03-21T12:52:17.180977+0000","last_clean":"2024-03-21T12:52:17.180977+0000","last_became_active":"2024-03-21T12:51:40.087646+0000","last_became_peered":"2024-03-21T12:51:40.087646+0000","last_unstale":"2024-03-21T12:52:17.180977+0000","last_undegraded":"2024-03-21T12:52:17.180977+0000","last_fullsized":"2024-03-21T12:52:17.180977+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:39:08.769765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.049710+0000","last_change":"2024-03-21T12:51:36.074573+0000","last_active":"2024-03-21T12:52:17.049710+0000","last_peered":"2024-03-21T12:52:17.049710+0000","last_clean":"2024-03-21T12:52:17.049710+0000","last_became_active":"2024-03-21T12:51:36.073984+0000","last_became_peered":"2024-03-21T12:51:36.073984+0000","last_unstale":"2024-03-21T12:52:17.049710+0000","last_undegraded":"2024-03-21T12:52:17.049710+0000","last_fullsized":"2024-03-21T12:52:17.049710+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:22:44.877000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"69'11","reported_seq":48,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180776+0000","last_change":"2024-03-21T12:51:38.092551+0000","last_active":"2024-03-21T12:52:17.180776+0000","last_peered":"2024-03-21T12:52:17.180776+0000","last_clean":"2024-03-21T12:52:17.180776+0000","last_became_active":"2024-03-21T12:51:38.092200+0000","last_became_peered":"2024-03-21T12:51:38.092200+0000","last_unstale":"2024-03-21T12:52:17.180776+0000","last_undegraded":"2024-03-21T12:52:17.180776+0000","last_fullsized":"2024-03-21T12:52:17.180776+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:03:40.189935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.163345+0000","last_change":"2024-03-21T12:51:42.091573+0000","last_active":"2024-03-21T12:52:17.163345+0000","last_peered":"2024-03-21T12:52:17.163345+0000","last_clean":"2024-03-21T12:52:17.163345+0000","last_became_active":"2024-03-21T12:51:42.091194+0000","last_became_peered":"2024-03-21T12:51:42.091194+0000","last_unstale":"2024-03-21T12:52:17.163345+0000","last_undegraded":"2024-03-21T12:52:17.163345+0000","last_fullsized":"2024-03-21T12:52:17.163345+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:34:24.166192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179961+0000","last_change":"2024-03-21T12:51:40.087634+0000","last_active":"2024-03-21T12:52:17.179961+0000","last_peered":"2024-03-21T12:52:17.179961+0000","last_clean":"2024-03-21T12:52:17.179961+0000","last_became_active":"2024-03-21T12:51:40.087331+0000","last_became_peered":"2024-03-21T12:51:40.087331+0000","last_unstale":"2024-03-21T12:52:17.179961+0000","last_undegraded":"2024-03-21T12:52:17.179961+0000","last_fullsized":"2024-03-21T12:52:17.179961+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:58:54.841811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180074+0000","last_change":"2024-03-21T12:51:36.078841+0000","last_active":"2024-03-21T12:52:17.180074+0000","last_peered":"2024-03-21T12:52:17.180074+0000","last_clean":"2024-03-21T12:52:17.180074+0000","last_became_active":"2024-03-21T12:51:36.078533+0000","last_became_peered":"2024-03-21T12:51:36.078533+0000","last_unstale":"2024-03-21T12:52:17.180074+0000","last_undegraded":"2024-03-21T12:52:17.180074+0000","last_fullsized":"2024-03-21T12:52:17.180074+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:39:05.360528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"69'4","reported_seq":35,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.049814+0000","last_change":"2024-03-21T12:51:38.087208+0000","last_active":"2024-03-21T12:52:17.049814+0000","last_peered":"2024-03-21T12:52:17.049814+0000","last_clean":"2024-03-21T12:52:17.049814+0000","last_became_active":"2024-03-21T12:51:38.086879+0000","last_became_peered":"2024-03-21T12:51:38.086879+0000","last_unstale":"2024-03-21T12:52:17.049814+0000","last_undegraded":"2024-03-21T12:52:17.049814+0000","last_fullsized":"2024-03-21T12:52:17.049814+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:56:24.378008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.050668+0000","last_change":"2024-03-21T12:51:42.180358+0000","last_active":"2024-03-21T12:52:17.050668+0000","last_peered":"2024-03-21T12:52:17.050668+0000","last_clean":"2024-03-21T12:52:17.050668+0000","last_became_active":"2024-03-21T12:51:42.180036+0000","last_became_peered":"2024-03-21T12:51:42.180036+0000","last_unstale":"2024-03-21T12:52:17.050668+0000","last_undegraded":"2024-03-21T12:52:17.050668+0000","last_fullsized":"2024-03-21T12:52:17.050668+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:30:44.704138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.163990+0000","last_change":"2024-03-21T12:51:40.090670+0000","last_active":"2024-03-21T12:52:17.163990+0000","last_peered":"2024-03-21T12:52:17.163990+0000","last_clean":"2024-03-21T12:52:17.163990+0000","last_became_active":"2024-03-21T12:51:40.090376+0000","last_became_peered":"2024-03-21T12:51:40.090376+0000","last_unstale":"2024-03-21T12:52:17.163990+0000","last_undegraded":"2024-03-21T12:52:17.163990+0000","last_fullsized":"2024-03-21T12:52:17.163990+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:35:48.022163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157641+0000","last_change":"2024-03-21T12:51:36.071020+0000","last_active":"2024-03-21T12:52:17.157641+0000","last_peered":"2024-03-21T12:52:17.157641+0000","last_clean":"2024-03-21T12:52:17.157641+0000","last_became_active":"2024-03-21T12:51:36.069867+0000","last_became_peered":"2024-03-21T12:51:36.069867+0000","last_unstale":"2024-03-21T12:52:17.157641+0000","last_undegraded":"2024-03-21T12:52:17.157641+0000","last_fullsized":"2024-03-21T12:52:17.157641+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:22:53.953657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"69'11","reported_seq":48,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.050644+0000","last_change":"2024-03-21T12:51:38.218405+0000","last_active":"2024-03-21T12:52:17.050644+0000","last_peered":"2024-03-21T12:52:17.050644+0000","last_clean":"2024-03-21T12:52:17.050644+0000","last_became_active":"2024-03-21T12:51:38.217937+0000","last_became_peered":"2024-03-21T12:51:38.217937+0000","last_unstale":"2024-03-21T12:52:17.050644+0000","last_undegraded":"2024-03-21T12:52:17.050644+0000","last_fullsized":"2024-03-21T12:52:17.050644+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:33:11.198452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179437+0000","last_change":"2024-03-21T12:51:42.096995+0000","last_active":"2024-03-21T12:52:17.179437+0000","last_peered":"2024-03-21T12:52:17.179437+0000","last_clean":"2024-03-21T12:52:17.179437+0000","last_became_active":"2024-03-21T12:51:42.096458+0000","last_became_peered":"2024-03-21T12:51:42.096458+0000","last_unstale":"2024-03-21T12:52:17.179437+0000","last_undegraded":"2024-03-21T12:52:17.179437+0000","last_fullsized":"2024-03-21T12:52:17.179437+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:50:31.751024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.050956+0000","last_change":"2024-03-21T12:51:40.093003+0000","last_active":"2024-03-21T12:52:17.050956+0000","last_peered":"2024-03-21T12:52:17.050956+0000","last_clean":"2024-03-21T12:52:17.050956+0000","last_became_active":"2024-03-21T12:51:40.092698+0000","last_became_peered":"2024-03-21T12:51:40.092698+0000","last_unstale":"2024-03-21T12:52:17.050956+0000","last_undegraded":"2024-03-21T12:52:17.050956+0000","last_fullsized":"2024-03-21T12:52:17.050956+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:09:45.273067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.163904+0000","last_change":"2024-03-21T12:51:36.077229+0000","last_active":"2024-03-21T12:52:17.163904+0000","last_peered":"2024-03-21T12:52:17.163904+0000","last_clean":"2024-03-21T12:52:17.163904+0000","last_became_active":"2024-03-21T12:51:36.076932+0000","last_became_peered":"2024-03-21T12:51:36.076932+0000","last_unstale":"2024-03-21T12:52:17.163904+0000","last_undegraded":"2024-03-21T12:52:17.163904+0000","last_fullsized":"2024-03-21T12:52:17.163904+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:43:20.705557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"69'9","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179889+0000","last_change":"2024-03-21T12:51:38.091052+0000","last_active":"2024-03-21T12:52:17.179889+0000","last_peered":"2024-03-21T12:52:17.179889+0000","last_clean":"2024-03-21T12:52:17.179889+0000","last_became_active":"2024-03-21T12:51:38.090694+0000","last_became_peered":"2024-03-21T12:51:38.090694+0000","last_unstale":"2024-03-21T12:52:17.179889+0000","last_undegraded":"2024-03-21T12:52:17.179889+0000","last_fullsized":"2024-03-21T12:52:17.179889+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:03:06.920096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.050300+0000","last_change":"2024-03-21T12:51:42.098826+0000","last_active":"2024-03-21T12:52:17.050300+0000","last_peered":"2024-03-21T12:52:17.050300+0000","last_clean":"2024-03-21T12:52:17.050300+0000","last_became_active":"2024-03-21T12:51:42.098535+0000","last_became_peered":"2024-03-21T12:51:42.098535+0000","last_unstale":"2024-03-21T12:52:17.050300+0000","last_undegraded":"2024-03-21T12:52:17.050300+0000","last_fullsized":"2024-03-21T12:52:17.050300+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:56:41.581909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179840+0000","last_change":"2024-03-21T12:51:40.090680+0000","last_active":"2024-03-21T12:52:17.179840+0000","last_peered":"2024-03-21T12:52:17.179840+0000","last_clean":"2024-03-21T12:52:17.179840+0000","last_became_active":"2024-03-21T12:51:40.090348+0000","last_became_peered":"2024-03-21T12:51:40.090348+0000","last_unstale":"2024-03-21T12:52:17.179840+0000","last_undegraded":"2024-03-21T12:52:17.179840+0000","last_fullsized":"2024-03-21T12:52:17.179840+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:10:21.164408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.052572+0000","last_change":"2024-03-21T12:51:36.076452+0000","last_active":"2024-03-21T12:52:17.052572+0000","last_peered":"2024-03-21T12:52:17.052572+0000","last_clean":"2024-03-21T12:52:17.052572+0000","last_became_active":"2024-03-21T12:51:36.076025+0000","last_became_peered":"2024-03-21T12:51:36.076025+0000","last_unstale":"2024-03-21T12:52:17.052572+0000","last_undegraded":"2024-03-21T12:52:17.052572+0000","last_fullsized":"2024-03-21T12:52:17.052572+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:38:34.645681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"69'11","reported_seq":48,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.052403+0000","last_change":"2024-03-21T12:51:38.088963+0000","last_active":"2024-03-21T12:52:17.052403+0000","last_peered":"2024-03-21T12:52:17.052403+0000","last_clean":"2024-03-21T12:52:17.052403+0000","last_became_active":"2024-03-21T12:51:38.088485+0000","last_became_peered":"2024-03-21T12:51:38.088485+0000","last_unstale":"2024-03-21T12:52:17.052403+0000","last_undegraded":"2024-03-21T12:52:17.052403+0000","last_fullsized":"2024-03-21T12:52:17.052403+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:12:17.857435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180278+0000","last_change":"2024-03-21T12:51:42.098840+0000","last_active":"2024-03-21T12:52:17.180278+0000","last_peered":"2024-03-21T12:52:17.180278+0000","last_clean":"2024-03-21T12:52:17.180278+0000","last_became_active":"2024-03-21T12:51:42.098508+0000","last_became_peered":"2024-03-21T12:51:42.098508+0000","last_unstale":"2024-03-21T12:52:17.180278+0000","last_undegraded":"2024-03-21T12:52:17.180278+0000","last_fullsized":"2024-03-21T12:52:17.180278+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:08:23.656888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"69'8","reported_seq":50,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:51.132508+0000","last_change":"2024-03-21T12:51:40.094253+0000","last_active":"2024-03-21T12:52:51.132508+0000","last_peered":"2024-03-21T12:52:51.132508+0000","last_clean":"2024-03-21T12:52:51.132508+0000","last_became_active":"2024-03-21T12:51:40.093585+0000","last_became_peered":"2024-03-21T12:51:40.093585+0000","last_unstale":"2024-03-21T12:52:51.132508+0000","last_undegraded":"2024-03-21T12:52:51.132508+0000","last_fullsized":"2024-03-21T12:52:51.132508+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:08:38.727761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.163687+0000","last_change":"2024-03-21T12:51:36.079003+0000","last_active":"2024-03-21T12:52:17.163687+0000","last_peered":"2024-03-21T12:52:17.163687+0000","last_clean":"2024-03-21T12:52:17.163687+0000","last_became_active":"2024-03-21T12:51:36.078625+0000","last_became_peered":"2024-03-21T12:51:36.078625+0000","last_unstale":"2024-03-21T12:52:17.163687+0000","last_undegraded":"2024-03-21T12:52:17.163687+0000","last_fullsized":"2024-03-21T12:52:17.163687+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:14:22.361186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"69'10","reported_seq":44,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.050478+0000","last_change":"2024-03-21T12:51:38.217895+0000","last_active":"2024-03-21T12:52:17.050478+0000","last_peered":"2024-03-21T12:52:17.050478+0000","last_clean":"2024-03-21T12:52:17.050478+0000","last_became_active":"2024-03-21T12:51:38.217176+0000","last_became_peered":"2024-03-21T12:51:38.217176+0000","last_unstale":"2024-03-21T12:52:17.050478+0000","last_undegraded":"2024-03-21T12:52:17.050478+0000","last_fullsized":"2024-03-21T12:52:17.050478+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:42:20.714369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.053100+0000","last_change":"2024-03-21T12:51:42.093000+0000","last_active":"2024-03-21T12:52:17.053100+0000","last_peered":"2024-03-21T12:52:17.053100+0000","last_clean":"2024-03-21T12:52:17.053100+0000","last_became_active":"2024-03-21T12:51:42.092582+0000","last_became_peered":"2024-03-21T12:51:42.092582+0000","last_unstale":"2024-03-21T12:52:17.053100+0000","last_undegraded":"2024-03-21T12:52:17.053100+0000","last_fullsized":"2024-03-21T12:52:17.053100+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:51:06.548012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"69'8","reported_seq":50,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:51.132154+0000","last_change":"2024-03-21T12:51:40.094130+0000","last_active":"2024-03-21T12:52:51.132154+0000","last_peered":"2024-03-21T12:52:51.132154+0000","last_clean":"2024-03-21T12:52:51.132154+0000","last_became_active":"2024-03-21T12:51:40.093765+0000","last_became_peered":"2024-03-21T12:51:40.093765+0000","last_unstale":"2024-03-21T12:52:51.132154+0000","last_undegraded":"2024-03-21T12:52:51.132154+0000","last_fullsized":"2024-03-21T12:52:51.132154+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:20:42.520650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179910+0000","last_change":"2024-03-21T12:51:36.077294+0000","last_active":"2024-03-21T12:52:17.179910+0000","last_peered":"2024-03-21T12:52:17.179910+0000","last_clean":"2024-03-21T12:52:17.179910+0000","last_became_active":"2024-03-21T12:51:36.076905+0000","last_became_peered":"2024-03-21T12:51:36.076905+0000","last_unstale":"2024-03-21T12:52:17.179910+0000","last_undegraded":"2024-03-21T12:52:17.179910+0000","last_fullsized":"2024-03-21T12:52:17.179910+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:54:29.026133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"69'9","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157806+0000","last_change":"2024-03-21T12:51:38.218797+0000","last_active":"2024-03-21T12:52:17.157806+0000","last_peered":"2024-03-21T12:52:17.157806+0000","last_clean":"2024-03-21T12:52:17.157806+0000","last_became_active":"2024-03-21T12:51:38.218300+0000","last_became_peered":"2024-03-21T12:51:38.218300+0000","last_unstale":"2024-03-21T12:52:17.157806+0000","last_undegraded":"2024-03-21T12:52:17.157806+0000","last_fullsized":"2024-03-21T12:52:17.157806+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:30:19.063311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.050403+0000","last_change":"2024-03-21T12:51:42.090614+0000","last_active":"2024-03-21T12:52:17.050403+0000","last_peered":"2024-03-21T12:52:17.050403+0000","last_clean":"2024-03-21T12:52:17.050403+0000","last_became_active":"2024-03-21T12:51:42.090264+0000","last_became_peered":"2024-03-21T12:51:42.090264+0000","last_unstale":"2024-03-21T12:52:17.050403+0000","last_undegraded":"2024-03-21T12:52:17.050403+0000","last_fullsized":"2024-03-21T12:52:17.050403+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:39:15.150234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051618+0000","last_change":"2024-03-21T12:51:40.088526+0000","last_active":"2024-03-21T12:52:17.051618+0000","last_peered":"2024-03-21T12:52:17.051618+0000","last_clean":"2024-03-21T12:52:17.051618+0000","last_became_active":"2024-03-21T12:51:40.087842+0000","last_became_peered":"2024-03-21T12:51:40.087842+0000","last_unstale":"2024-03-21T12:52:17.051618+0000","last_undegraded":"2024-03-21T12:52:17.051618+0000","last_fullsized":"2024-03-21T12:52:17.051618+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:58:14.921052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.163479+0000","last_change":"2024-03-21T12:51:36.073263+0000","last_active":"2024-03-21T12:52:17.163479+0000","last_peered":"2024-03-21T12:52:17.163479+0000","last_clean":"2024-03-21T12:52:17.163479+0000","last_became_active":"2024-03-21T12:51:36.072946+0000","last_became_peered":"2024-03-21T12:51:36.072946+0000","last_unstale":"2024-03-21T12:52:17.163479+0000","last_undegraded":"2024-03-21T12:52:17.163479+0000","last_fullsized":"2024-03-21T12:52:17.163479+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:55:08.708449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"69'9","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180781+0000","last_change":"2024-03-21T12:51:38.219245+0000","last_active":"2024-03-21T12:52:17.180781+0000","last_peered":"2024-03-21T12:52:17.180781+0000","last_clean":"2024-03-21T12:52:17.180781+0000","last_became_active":"2024-03-21T12:51:38.218894+0000","last_became_peered":"2024-03-21T12:51:38.218894+0000","last_unstale":"2024-03-21T12:52:17.180781+0000","last_undegraded":"2024-03-21T12:52:17.180781+0000","last_fullsized":"2024-03-21T12:52:17.180781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:03:21.235285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.163723+0000","last_change":"2024-03-21T12:51:42.181552+0000","last_active":"2024-03-21T12:52:17.163723+0000","last_peered":"2024-03-21T12:52:17.163723+0000","last_clean":"2024-03-21T12:52:17.163723+0000","last_became_active":"2024-03-21T12:51:42.181261+0000","last_became_peered":"2024-03-21T12:51:42.181261+0000","last_unstale":"2024-03-21T12:52:17.163723+0000","last_undegraded":"2024-03-21T12:52:17.163723+0000","last_fullsized":"2024-03-21T12:52:17.163723+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:26:33.326732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157703+0000","last_change":"2024-03-21T12:51:40.092939+0000","last_active":"2024-03-21T12:52:17.157703+0000","last_peered":"2024-03-21T12:52:17.157703+0000","last_clean":"2024-03-21T12:52:17.157703+0000","last_became_active":"2024-03-21T12:51:40.092507+0000","last_became_peered":"2024-03-21T12:51:40.092507+0000","last_unstale":"2024-03-21T12:52:17.157703+0000","last_undegraded":"2024-03-21T12:52:17.157703+0000","last_fullsized":"2024-03-21T12:52:17.157703+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:14:27.750397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"62'1","reported_seq":39,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179887+0000","last_change":"2024-03-21T12:51:36.072251+0000","last_active":"2024-03-21T12:52:17.179887+0000","last_peered":"2024-03-21T12:52:17.179887+0000","last_clean":"2024-03-21T12:52:17.179887+0000","last_became_active":"2024-03-21T12:51:36.071958+0000","last_became_peered":"2024-03-21T12:51:36.071958+0000","last_unstale":"2024-03-21T12:52:17.179887+0000","last_undegraded":"2024-03-21T12:52:17.179887+0000","last_fullsized":"2024-03-21T12:52:17.179887+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:23:27.957768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":911,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"69'6","reported_seq":38,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051802+0000","last_change":"2024-03-21T12:51:38.217923+0000","last_active":"2024-03-21T12:52:17.051802+0000","last_peered":"2024-03-21T12:52:17.051802+0000","last_clean":"2024-03-21T12:52:17.051802+0000","last_became_active":"2024-03-21T12:51:38.217169+0000","last_became_peered":"2024-03-21T12:51:38.217169+0000","last_unstale":"2024-03-21T12:52:17.051802+0000","last_undegraded":"2024-03-21T12:52:17.051802+0000","last_fullsized":"2024-03-21T12:52:17.051802+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:29:57.972260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"69'1","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157296+0000","last_change":"2024-03-21T12:51:42.180920+0000","last_active":"2024-03-21T12:52:17.157296+0000","last_peered":"2024-03-21T12:52:17.157296+0000","last_clean":"2024-03-21T12:52:17.157296+0000","last_became_active":"2024-03-21T12:51:42.180578+0000","last_became_peered":"2024-03-21T12:51:42.180578+0000","last_unstale":"2024-03-21T12:52:17.157296+0000","last_undegraded":"2024-03-21T12:52:17.157296+0000","last_fullsized":"2024-03-21T12:52:17.157296+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:48:58.686198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"62'2","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.163975+0000","last_change":"2024-03-21T12:51:36.077977+0000","last_active":"2024-03-21T12:52:17.163975+0000","last_peered":"2024-03-21T12:52:17.163975+0000","last_clean":"2024-03-21T12:52:17.163975+0000","last_became_active":"2024-03-21T12:51:36.077661+0000","last_became_peered":"2024-03-21T12:51:36.077661+0000","last_unstale":"2024-03-21T12:52:17.163975+0000","last_undegraded":"2024-03-21T12:52:17.163975+0000","last_fullsized":"2024-03-21T12:52:17.163975+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:28:13.334015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"69'15","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.156997+0000","last_change":"2024-03-21T12:51:38.219468+0000","last_active":"2024-03-21T12:52:17.156997+0000","last_peered":"2024-03-21T12:52:17.156997+0000","last_clean":"2024-03-21T12:52:17.156997+0000","last_became_active":"2024-03-21T12:51:38.219125+0000","last_became_peered":"2024-03-21T12:51:38.219125+0000","last_unstale":"2024-03-21T12:52:17.156997+0000","last_undegraded":"2024-03-21T12:52:17.156997+0000","last_fullsized":"2024-03-21T12:52:17.156997+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:49:43.933463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"69'8","reported_seq":50,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:51.132496+0000","last_change":"2024-03-21T12:51:40.090707+0000","last_active":"2024-03-21T12:52:51.132496+0000","last_peered":"2024-03-21T12:52:51.132496+0000","last_clean":"2024-03-21T12:52:51.132496+0000","last_became_active":"2024-03-21T12:51:40.090408+0000","last_became_peered":"2024-03-21T12:51:40.090408+0000","last_unstale":"2024-03-21T12:52:51.132496+0000","last_undegraded":"2024-03-21T12:52:51.132496+0000","last_fullsized":"2024-03-21T12:52:51.132496+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:38:02.647098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.050774+0000","last_change":"2024-03-21T12:51:42.095706+0000","last_active":"2024-03-21T12:52:17.050774+0000","last_peered":"2024-03-21T12:52:17.050774+0000","last_clean":"2024-03-21T12:52:17.050774+0000","last_became_active":"2024-03-21T12:51:42.094206+0000","last_became_peered":"2024-03-21T12:51:42.094206+0000","last_unstale":"2024-03-21T12:52:17.050774+0000","last_undegraded":"2024-03-21T12:52:17.050774+0000","last_fullsized":"2024-03-21T12:52:17.050774+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:09:35.491438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.164011+0000","last_change":"2024-03-21T12:51:36.070333+0000","last_active":"2024-03-21T12:52:17.164011+0000","last_peered":"2024-03-21T12:52:17.164011+0000","last_clean":"2024-03-21T12:52:17.164011+0000","last_became_active":"2024-03-21T12:51:36.070042+0000","last_became_peered":"2024-03-21T12:51:36.070042+0000","last_unstale":"2024-03-21T12:52:17.164011+0000","last_undegraded":"2024-03-21T12:52:17.164011+0000","last_fullsized":"2024-03-21T12:52:17.164011+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:58:39.629607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"69'12","reported_seq":70,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.052158+0000","last_change":"2024-03-21T12:51:52.998558+0000","last_active":"2024-03-21T12:52:17.052158+0000","last_peered":"2024-03-21T12:52:17.052158+0000","last_clean":"2024-03-21T12:52:17.052158+0000","last_became_active":"2024-03-21T12:51:52.991832+0000","last_became_peered":"2024-03-21T12:51:52.991832+0000","last_unstale":"2024-03-21T12:52:17.052158+0000","last_undegraded":"2024-03-21T12:52:17.052158+0000","last_fullsized":"2024-03-21T12:52:17.052158+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:41:43.577719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,3],"acting":[4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.048207+0000","last_change":"2024-03-21T12:51:40.085820+0000","last_active":"2024-03-21T12:52:17.048207+0000","last_peered":"2024-03-21T12:52:17.048207+0000","last_clean":"2024-03-21T12:52:17.048207+0000","last_became_active":"2024-03-21T12:51:40.085537+0000","last_became_peered":"2024-03-21T12:51:40.085537+0000","last_unstale":"2024-03-21T12:52:17.048207+0000","last_undegraded":"2024-03-21T12:52:17.048207+0000","last_fullsized":"2024-03-21T12:52:17.048207+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:30:01.662575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.163215+0000","last_change":"2024-03-21T12:51:42.091977+0000","last_active":"2024-03-21T12:52:17.163215+0000","last_peered":"2024-03-21T12:52:17.163215+0000","last_clean":"2024-03-21T12:52:17.163215+0000","last_became_active":"2024-03-21T12:51:42.091290+0000","last_became_peered":"2024-03-21T12:51:42.091290+0000","last_unstale":"2024-03-21T12:52:17.163215+0000","last_undegraded":"2024-03-21T12:52:17.163215+0000","last_fullsized":"2024-03-21T12:52:17.163215+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:11:31.592650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.048417+0000","last_change":"2024-03-21T12:51:40.091513+0000","last_active":"2024-03-21T12:52:17.048417+0000","last_peered":"2024-03-21T12:52:17.048417+0000","last_clean":"2024-03-21T12:52:17.048417+0000","last_became_active":"2024-03-21T12:51:40.091151+0000","last_became_peered":"2024-03-21T12:51:40.091151+0000","last_unstale":"2024-03-21T12:52:17.048417+0000","last_undegraded":"2024-03-21T12:52:17.048417+0000","last_fullsized":"2024-03-21T12:52:17.048417+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:46:16.217484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.164162+0000","last_change":"2024-03-21T12:51:36.074462+0000","last_active":"2024-03-21T12:52:17.164162+0000","last_peered":"2024-03-21T12:52:17.164162+0000","last_clean":"2024-03-21T12:52:17.164162+0000","last_became_active":"2024-03-21T12:51:36.074122+0000","last_became_peered":"2024-03-21T12:51:36.074122+0000","last_unstale":"2024-03-21T12:52:17.164162+0000","last_undegraded":"2024-03-21T12:52:17.164162+0000","last_fullsized":"2024-03-21T12:52:17.164162+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:14:16.354037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"69'19","reported_seq":60,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180184+0000","last_change":"2024-03-21T12:51:38.218339+0000","last_active":"2024-03-21T12:52:17.180184+0000","last_peered":"2024-03-21T12:52:17.180184+0000","last_clean":"2024-03-21T12:52:17.180184+0000","last_became_active":"2024-03-21T12:51:38.217932+0000","last_became_peered":"2024-03-21T12:51:38.217932+0000","last_unstale":"2024-03-21T12:52:17.180184+0000","last_undegraded":"2024-03-21T12:52:17.180184+0000","last_fullsized":"2024-03-21T12:52:17.180184+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:18:42.725674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179536+0000","last_change":"2024-03-21T12:51:42.095439+0000","last_active":"2024-03-21T12:52:17.179536+0000","last_peered":"2024-03-21T12:52:17.179536+0000","last_clean":"2024-03-21T12:52:17.179536+0000","last_became_active":"2024-03-21T12:51:42.095096+0000","last_became_peered":"2024-03-21T12:51:42.095096+0000","last_unstale":"2024-03-21T12:52:17.179536+0000","last_undegraded":"2024-03-21T12:52:17.179536+0000","last_fullsized":"2024-03-21T12:52:17.179536+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:20:53.350200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.049694+0000","last_change":"2024-03-21T12:51:40.088305+0000","last_active":"2024-03-21T12:52:17.049694+0000","last_peered":"2024-03-21T12:52:17.049694+0000","last_clean":"2024-03-21T12:52:17.049694+0000","last_became_active":"2024-03-21T12:51:40.087907+0000","last_became_peered":"2024-03-21T12:51:40.087907+0000","last_unstale":"2024-03-21T12:52:17.049694+0000","last_undegraded":"2024-03-21T12:52:17.049694+0000","last_fullsized":"2024-03-21T12:52:17.049694+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:40.168333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157344+0000","last_change":"2024-03-21T12:51:36.069544+0000","last_active":"2024-03-21T12:52:17.157344+0000","last_peered":"2024-03-21T12:52:17.157344+0000","last_clean":"2024-03-21T12:52:17.157344+0000","last_became_active":"2024-03-21T12:51:36.069231+0000","last_became_peered":"2024-03-21T12:51:36.069231+0000","last_unstale":"2024-03-21T12:52:17.157344+0000","last_undegraded":"2024-03-21T12:52:17.157344+0000","last_fullsized":"2024-03-21T12:52:17.157344+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:46:57.430679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"69'9","reported_seq":45,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.181386+0000","last_change":"2024-03-21T12:51:38.085970+0000","last_active":"2024-03-21T12:52:17.181386+0000","last_peered":"2024-03-21T12:52:17.181386+0000","last_clean":"2024-03-21T12:52:17.181386+0000","last_became_active":"2024-03-21T12:51:38.085526+0000","last_became_peered":"2024-03-21T12:51:38.085526+0000","last_unstale":"2024-03-21T12:52:17.181386+0000","last_undegraded":"2024-03-21T12:52:17.181386+0000","last_fullsized":"2024-03-21T12:52:17.181386+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:02:33.907384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051103+0000","last_change":"2024-03-21T12:51:42.101403+0000","last_active":"2024-03-21T12:52:17.051103+0000","last_peered":"2024-03-21T12:52:17.051103+0000","last_clean":"2024-03-21T12:52:17.051103+0000","last_became_active":"2024-03-21T12:51:42.100887+0000","last_became_peered":"2024-03-21T12:51:42.100887+0000","last_unstale":"2024-03-21T12:52:17.051103+0000","last_undegraded":"2024-03-21T12:52:17.051103+0000","last_fullsized":"2024-03-21T12:52:17.051103+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:35:52.065907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"65'2","reported_seq":44,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157018+0000","last_change":"2024-03-21T12:51:39.077942+0000","last_active":"2024-03-21T12:52:17.157018+0000","last_peered":"2024-03-21T12:52:17.157018+0000","last_clean":"2024-03-21T12:52:17.157018+0000","last_became_active":"2024-03-21T12:51:35.065288+0000","last_became_peered":"2024-03-21T12:51:35.065288+0000","last_unstale":"2024-03-21T12:52:17.157018+0000","last_undegraded":"2024-03-21T12:52:17.157018+0000","last_fullsized":"2024-03-21T12:52:17.157018+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:34.048471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:34.048471+0000","last_clean_scrub_stamp":"2024-03-21T12:51:34.048471+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:38:07.675192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001525169,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051957+0000","last_change":"2024-03-21T12:51:36.075601+0000","last_active":"2024-03-21T12:52:17.051957+0000","last_peered":"2024-03-21T12:52:17.051957+0000","last_clean":"2024-03-21T12:52:17.051957+0000","last_became_active":"2024-03-21T12:51:36.075231+0000","last_became_peered":"2024-03-21T12:51:36.075231+0000","last_unstale":"2024-03-21T12:52:17.051957+0000","last_undegraded":"2024-03-21T12:52:17.051957+0000","last_fullsized":"2024-03-21T12:52:17.051957+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:22:34.166997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"69'28","reported_seq":94,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:51.132095+0000","last_change":"2024-03-21T12:51:38.089283+0000","last_active":"2024-03-21T12:52:51.132095+0000","last_peered":"2024-03-21T12:52:51.132095+0000","last_clean":"2024-03-21T12:52:51.132095+0000","last_became_active":"2024-03-21T12:51:38.088682+0000","last_became_peered":"2024-03-21T12:51:38.088682+0000","last_unstale":"2024-03-21T12:52:51.132095+0000","last_undegraded":"2024-03-21T12:52:51.132095+0000","last_fullsized":"2024-03-21T12:52:51.132095+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:45:22.774914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180094+0000","last_change":"2024-03-21T12:51:40.088236+0000","last_active":"2024-03-21T12:52:17.180094+0000","last_peered":"2024-03-21T12:52:17.180094+0000","last_clean":"2024-03-21T12:52:17.180094+0000","last_became_active":"2024-03-21T12:51:40.086146+0000","last_became_peered":"2024-03-21T12:51:40.086146+0000","last_unstale":"2024-03-21T12:52:17.180094+0000","last_undegraded":"2024-03-21T12:52:17.180094+0000","last_fullsized":"2024-03-21T12:52:17.180094+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:29:43.587618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.164261+0000","last_change":"2024-03-21T12:51:42.180753+0000","last_active":"2024-03-21T12:52:17.164261+0000","last_peered":"2024-03-21T12:52:17.164261+0000","last_clean":"2024-03-21T12:52:17.164261+0000","last_became_active":"2024-03-21T12:51:42.180443+0000","last_became_peered":"2024-03-21T12:51:42.180443+0000","last_unstale":"2024-03-21T12:52:17.164261+0000","last_undegraded":"2024-03-21T12:52:17.164261+0000","last_fullsized":"2024-03-21T12:52:17.164261+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:36:29.737201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179567+0000","last_change":"2024-03-21T12:51:36.073957+0000","last_active":"2024-03-21T12:52:17.179567+0000","last_peered":"2024-03-21T12:52:17.179567+0000","last_clean":"2024-03-21T12:52:17.179567+0000","last_became_active":"2024-03-21T12:51:36.073553+0000","last_became_peered":"2024-03-21T12:51:36.073553+0000","last_unstale":"2024-03-21T12:52:17.179567+0000","last_undegraded":"2024-03-21T12:52:17.179567+0000","last_fullsized":"2024-03-21T12:52:17.179567+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:28:43.225482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":42,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.049179+0000","last_change":"2024-03-21T12:51:39.074088+0000","last_active":"2024-03-21T12:52:17.049179+0000","last_peered":"2024-03-21T12:52:17.049179+0000","last_clean":"2024-03-21T12:52:17.049179+0000","last_became_active":"2024-03-21T12:51:35.064091+0000","last_became_peered":"2024-03-21T12:51:35.064091+0000","last_unstale":"2024-03-21T12:52:17.049179+0000","last_undegraded":"2024-03-21T12:52:17.049179+0000","last_fullsized":"2024-03-21T12:52:17.049179+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:34.048471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:34.048471+0000","last_clean_scrub_stamp":"2024-03-21T12:51:34.048471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:04:28.184801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000457348,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"69'13","reported_seq":56,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180910+0000","last_change":"2024-03-21T12:51:38.090405+0000","last_active":"2024-03-21T12:52:17.180910+0000","last_peered":"2024-03-21T12:52:17.180910+0000","last_clean":"2024-03-21T12:52:17.180910+0000","last_became_active":"2024-03-21T12:51:38.090094+0000","last_became_peered":"2024-03-21T12:51:38.090094+0000","last_unstale":"2024-03-21T12:52:17.180910+0000","last_undegraded":"2024-03-21T12:52:17.180910+0000","last_fullsized":"2024-03-21T12:52:17.180910+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:56:23.536335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.048839+0000","last_change":"2024-03-21T12:51:40.094943+0000","last_active":"2024-03-21T12:52:17.048839+0000","last_peered":"2024-03-21T12:52:17.048839+0000","last_clean":"2024-03-21T12:52:17.048839+0000","last_became_active":"2024-03-21T12:51:40.092512+0000","last_became_peered":"2024-03-21T12:51:40.092512+0000","last_unstale":"2024-03-21T12:52:17.048839+0000","last_undegraded":"2024-03-21T12:52:17.048839+0000","last_fullsized":"2024-03-21T12:52:17.048839+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:00:54.524557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157678+0000","last_change":"2024-03-21T12:51:42.097990+0000","last_active":"2024-03-21T12:52:17.157678+0000","last_peered":"2024-03-21T12:52:17.157678+0000","last_clean":"2024-03-21T12:52:17.157678+0000","last_became_active":"2024-03-21T12:51:42.097683+0000","last_became_peered":"2024-03-21T12:51:42.097683+0000","last_unstale":"2024-03-21T12:52:17.157678+0000","last_undegraded":"2024-03-21T12:52:17.157678+0000","last_fullsized":"2024-03-21T12:52:17.157678+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:50:43.529755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"73'128","reported_seq":278,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:19.149615+0000","last_change":"2024-03-21T12:51:24.555056+0000","last_active":"2024-03-21T12:52:19.149615+0000","last_peered":"2024-03-21T12:52:19.149615+0000","last_clean":"2024-03-21T12:52:19.149615+0000","last_became_active":"2024-03-21T12:51:24.554651+0000","last_became_peered":"2024-03-21T12:51:24.554651+0000","last_unstale":"2024-03-21T12:52:19.149615+0000","last_undegraded":"2024-03-21T12:52:19.149615+0000","last_fullsized":"2024-03-21T12:52:19.149615+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:49:12.895321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:49:12.895321+0000","last_clean_scrub_stamp":"2024-03-21T12:49:12.895321+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:55:11.444449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.049670+0000","last_change":"2024-03-21T12:51:36.076902+0000","last_active":"2024-03-21T12:52:17.049670+0000","last_peered":"2024-03-21T12:52:17.049670+0000","last_clean":"2024-03-21T12:52:17.049670+0000","last_became_active":"2024-03-21T12:51:36.076579+0000","last_became_peered":"2024-03-21T12:51:36.076579+0000","last_unstale":"2024-03-21T12:52:17.049670+0000","last_undegraded":"2024-03-21T12:52:17.049670+0000","last_fullsized":"2024-03-21T12:52:17.049670+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:27:59.137686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"69'16","reported_seq":68,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:51.132737+0000","last_change":"2024-03-21T12:51:38.218501+0000","last_active":"2024-03-21T12:52:51.132737+0000","last_peered":"2024-03-21T12:52:51.132737+0000","last_clean":"2024-03-21T12:52:51.132737+0000","last_became_active":"2024-03-21T12:51:38.217927+0000","last_became_peered":"2024-03-21T12:51:38.217927+0000","last_unstale":"2024-03-21T12:52:51.132737+0000","last_undegraded":"2024-03-21T12:52:51.132737+0000","last_fullsized":"2024-03-21T12:52:51.132737+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:31:34.554432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.163819+0000","last_change":"2024-03-21T12:51:40.091597+0000","last_active":"2024-03-21T12:52:17.163819+0000","last_peered":"2024-03-21T12:52:17.163819+0000","last_clean":"2024-03-21T12:52:17.163819+0000","last_became_active":"2024-03-21T12:51:40.091293+0000","last_became_peered":"2024-03-21T12:51:40.091293+0000","last_unstale":"2024-03-21T12:52:17.163819+0000","last_undegraded":"2024-03-21T12:52:17.163819+0000","last_fullsized":"2024-03-21T12:52:17.163819+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:41:46.571271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180099+0000","last_change":"2024-03-21T12:51:42.090303+0000","last_active":"2024-03-21T12:52:17.180099+0000","last_peered":"2024-03-21T12:52:17.180099+0000","last_clean":"2024-03-21T12:52:17.180099+0000","last_became_active":"2024-03-21T12:51:42.089949+0000","last_became_peered":"2024-03-21T12:51:42.089949+0000","last_unstale":"2024-03-21T12:52:17.180099+0000","last_undegraded":"2024-03-21T12:52:17.180099+0000","last_fullsized":"2024-03-21T12:52:17.180099+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:15:49.504071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179738+0000","last_change":"2024-03-21T12:51:36.078285+0000","last_active":"2024-03-21T12:52:17.179738+0000","last_peered":"2024-03-21T12:52:17.179738+0000","last_clean":"2024-03-21T12:52:17.179738+0000","last_became_active":"2024-03-21T12:51:36.077980+0000","last_became_peered":"2024-03-21T12:51:36.077980+0000","last_unstale":"2024-03-21T12:52:17.179738+0000","last_undegraded":"2024-03-21T12:52:17.179738+0000","last_fullsized":"2024-03-21T12:52:17.179738+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:46:03.937522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"70'5","reported_seq":121,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:56.659422+0000","last_change":"2024-03-21T12:51:39.079533+0000","last_active":"2024-03-21T12:52:56.659422+0000","last_peered":"2024-03-21T12:52:56.659422+0000","last_clean":"2024-03-21T12:52:56.659422+0000","last_became_active":"2024-03-21T12:51:35.065822+0000","last_became_peered":"2024-03-21T12:51:35.065822+0000","last_unstale":"2024-03-21T12:52:56.659422+0000","last_undegraded":"2024-03-21T12:52:56.659422+0000","last_fullsized":"2024-03-21T12:52:56.659422+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:34.048471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:34.048471+0000","last_clean_scrub_stamp":"2024-03-21T12:51:34.048471+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:35:37.899070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013453339999999999,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"69'12","reported_seq":47,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180382+0000","last_change":"2024-03-21T12:51:38.084972+0000","last_active":"2024-03-21T12:52:17.180382+0000","last_peered":"2024-03-21T12:52:17.180382+0000","last_clean":"2024-03-21T12:52:17.180382+0000","last_became_active":"2024-03-21T12:51:38.084609+0000","last_became_peered":"2024-03-21T12:51:38.084609+0000","last_unstale":"2024-03-21T12:52:17.180382+0000","last_undegraded":"2024-03-21T12:52:17.180382+0000","last_fullsized":"2024-03-21T12:52:17.180382+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:22:06.342925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157907+0000","last_change":"2024-03-21T12:51:40.093166+0000","last_active":"2024-03-21T12:52:17.157907+0000","last_peered":"2024-03-21T12:52:17.157907+0000","last_clean":"2024-03-21T12:52:17.157907+0000","last_became_active":"2024-03-21T12:51:40.092876+0000","last_became_peered":"2024-03-21T12:51:40.092876+0000","last_unstale":"2024-03-21T12:52:17.157907+0000","last_undegraded":"2024-03-21T12:52:17.157907+0000","last_fullsized":"2024-03-21T12:52:17.157907+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:40:55.541743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179378+0000","last_change":"2024-03-21T12:51:42.179981+0000","last_active":"2024-03-21T12:52:17.179378+0000","last_peered":"2024-03-21T12:52:17.179378+0000","last_clean":"2024-03-21T12:52:17.179378+0000","last_became_active":"2024-03-21T12:51:42.179520+0000","last_became_peered":"2024-03-21T12:51:42.179520+0000","last_unstale":"2024-03-21T12:52:17.179378+0000","last_undegraded":"2024-03-21T12:52:17.179378+0000","last_fullsized":"2024-03-21T12:52:17.179378+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:59:35.003933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"62'1","reported_seq":39,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180208+0000","last_change":"2024-03-21T12:51:36.064904+0000","last_active":"2024-03-21T12:52:17.180208+0000","last_peered":"2024-03-21T12:52:17.180208+0000","last_clean":"2024-03-21T12:52:17.180208+0000","last_became_active":"2024-03-21T12:51:36.064600+0000","last_became_peered":"2024-03-21T12:51:36.064600+0000","last_unstale":"2024-03-21T12:52:17.180208+0000","last_undegraded":"2024-03-21T12:52:17.180208+0000","last_fullsized":"2024-03-21T12:52:17.180208+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:17:06.103623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"69'19","reported_seq":65,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180995+0000","last_change":"2024-03-21T12:51:38.217478+0000","last_active":"2024-03-21T12:52:17.180995+0000","last_peered":"2024-03-21T12:52:17.180995+0000","last_clean":"2024-03-21T12:52:17.180995+0000","last_became_active":"2024-03-21T12:51:38.217190+0000","last_became_peered":"2024-03-21T12:51:38.217190+0000","last_unstale":"2024-03-21T12:52:17.180995+0000","last_undegraded":"2024-03-21T12:52:17.180995+0000","last_fullsized":"2024-03-21T12:52:17.180995+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:04:13.088640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180273+0000","last_change":"2024-03-21T12:51:40.092003+0000","last_active":"2024-03-21T12:52:17.180273+0000","last_peered":"2024-03-21T12:52:17.180273+0000","last_clean":"2024-03-21T12:52:17.180273+0000","last_became_active":"2024-03-21T12:51:40.091738+0000","last_became_peered":"2024-03-21T12:51:40.091738+0000","last_unstale":"2024-03-21T12:52:17.180273+0000","last_undegraded":"2024-03-21T12:52:17.180273+0000","last_fullsized":"2024-03-21T12:52:17.180273+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:30:09.604700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.179565+0000","last_change":"2024-03-21T12:51:42.100627+0000","last_active":"2024-03-21T12:52:17.179565+0000","last_peered":"2024-03-21T12:52:17.179565+0000","last_clean":"2024-03-21T12:52:17.179565+0000","last_became_active":"2024-03-21T12:51:42.100281+0000","last_became_peered":"2024-03-21T12:51:42.100281+0000","last_unstale":"2024-03-21T12:52:17.179565+0000","last_undegraded":"2024-03-21T12:52:17.179565+0000","last_fullsized":"2024-03-21T12:52:17.179565+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:48:24.792167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157699+0000","last_change":"2024-03-21T12:51:36.070884+0000","last_active":"2024-03-21T12:52:17.157699+0000","last_peered":"2024-03-21T12:52:17.157699+0000","last_clean":"2024-03-21T12:52:17.157699+0000","last_became_active":"2024-03-21T12:51:36.069906+0000","last_became_peered":"2024-03-21T12:51:36.069906+0000","last_unstale":"2024-03-21T12:52:17.157699+0000","last_undegraded":"2024-03-21T12:52:17.157699+0000","last_fullsized":"2024-03-21T12:52:17.157699+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:31:35.512295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"69'10","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.049389+0000","last_change":"2024-03-21T12:51:52.999055+0000","last_active":"2024-03-21T12:52:17.049389+0000","last_peered":"2024-03-21T12:52:17.049389+0000","last_clean":"2024-03-21T12:52:17.049389+0000","last_became_active":"2024-03-21T12:51:52.992240+0000","last_became_peered":"2024-03-21T12:51:52.992240+0000","last_unstale":"2024-03-21T12:52:17.049389+0000","last_undegraded":"2024-03-21T12:52:17.049389+0000","last_fullsized":"2024-03-21T12:52:17.049389+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:08:06.576697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"69'8","reported_seq":50,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:51.132094+0000","last_change":"2024-03-21T12:51:40.091815+0000","last_active":"2024-03-21T12:52:51.132094+0000","last_peered":"2024-03-21T12:52:51.132094+0000","last_clean":"2024-03-21T12:52:51.132094+0000","last_became_active":"2024-03-21T12:51:40.091473+0000","last_became_peered":"2024-03-21T12:51:40.091473+0000","last_unstale":"2024-03-21T12:52:51.132094+0000","last_undegraded":"2024-03-21T12:52:51.132094+0000","last_fullsized":"2024-03-21T12:52:51.132094+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:38:46.914405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.164060+0000","last_change":"2024-03-21T12:51:42.181425+0000","last_active":"2024-03-21T12:52:17.164060+0000","last_peered":"2024-03-21T12:52:17.164060+0000","last_clean":"2024-03-21T12:52:17.164060+0000","last_became_active":"2024-03-21T12:51:42.181095+0000","last_became_peered":"2024-03-21T12:51:42.181095+0000","last_unstale":"2024-03-21T12:52:17.164060+0000","last_undegraded":"2024-03-21T12:52:17.164060+0000","last_fullsized":"2024-03-21T12:52:17.164060+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:51:39.385233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"62'1","reported_seq":34,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180535+0000","last_change":"2024-03-21T12:51:36.074641+0000","last_active":"2024-03-21T12:52:17.180535+0000","last_peered":"2024-03-21T12:52:17.180535+0000","last_clean":"2024-03-21T12:52:17.180535+0000","last_became_active":"2024-03-21T12:51:36.073664+0000","last_became_peered":"2024-03-21T12:51:36.073664+0000","last_unstale":"2024-03-21T12:52:17.180535+0000","last_undegraded":"2024-03-21T12:52:17.180535+0000","last_fullsized":"2024-03-21T12:52:17.180535+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T22:35:21.566345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"69'14","reported_seq":50,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051962+0000","last_change":"2024-03-21T12:51:38.086062+0000","last_active":"2024-03-21T12:52:17.051962+0000","last_peered":"2024-03-21T12:52:17.051962+0000","last_clean":"2024-03-21T12:52:17.051962+0000","last_became_active":"2024-03-21T12:51:38.085712+0000","last_became_peered":"2024-03-21T12:51:38.085712+0000","last_unstale":"2024-03-21T12:52:17.051962+0000","last_undegraded":"2024-03-21T12:52:17.051962+0000","last_fullsized":"2024-03-21T12:52:17.051962+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:20:26.791133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"69'8","reported_seq":50,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:51.132120+0000","last_change":"2024-03-21T12:51:40.097446+0000","last_active":"2024-03-21T12:52:51.132120+0000","last_peered":"2024-03-21T12:52:51.132120+0000","last_clean":"2024-03-21T12:52:51.132120+0000","last_became_active":"2024-03-21T12:51:40.096424+0000","last_became_peered":"2024-03-21T12:51:40.096424+0000","last_unstale":"2024-03-21T12:52:51.132120+0000","last_undegraded":"2024-03-21T12:52:51.132120+0000","last_fullsized":"2024-03-21T12:52:51.132120+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:52:47.609458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051972+0000","last_change":"2024-03-21T12:51:42.087579+0000","last_active":"2024-03-21T12:52:17.051972+0000","last_peered":"2024-03-21T12:52:17.051972+0000","last_clean":"2024-03-21T12:52:17.051972+0000","last_became_active":"2024-03-21T12:51:42.087267+0000","last_became_peered":"2024-03-21T12:51:42.087267+0000","last_unstale":"2024-03-21T12:52:17.051972+0000","last_undegraded":"2024-03-21T12:52:17.051972+0000","last_fullsized":"2024-03-21T12:52:17.051972+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:09:44.484937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051520+0000","last_change":"2024-03-21T12:51:36.078175+0000","last_active":"2024-03-21T12:52:17.051520+0000","last_peered":"2024-03-21T12:52:17.051520+0000","last_clean":"2024-03-21T12:52:17.051520+0000","last_became_active":"2024-03-21T12:51:36.077875+0000","last_became_peered":"2024-03-21T12:51:36.077875+0000","last_unstale":"2024-03-21T12:52:17.051520+0000","last_undegraded":"2024-03-21T12:52:17.051520+0000","last_fullsized":"2024-03-21T12:52:17.051520+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:53:47.186230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"69'18","reported_seq":61,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.050007+0000","last_change":"2024-03-21T12:51:38.217783+0000","last_active":"2024-03-21T12:52:17.050007+0000","last_peered":"2024-03-21T12:52:17.050007+0000","last_clean":"2024-03-21T12:52:17.050007+0000","last_became_active":"2024-03-21T12:51:38.217458+0000","last_became_peered":"2024-03-21T12:51:38.217458+0000","last_unstale":"2024-03-21T12:52:17.050007+0000","last_undegraded":"2024-03-21T12:52:17.050007+0000","last_fullsized":"2024-03-21T12:52:17.050007+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:17:33.669645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051798+0000","last_change":"2024-03-21T12:51:40.089429+0000","last_active":"2024-03-21T12:52:17.051798+0000","last_peered":"2024-03-21T12:52:17.051798+0000","last_clean":"2024-03-21T12:52:17.051798+0000","last_became_active":"2024-03-21T12:51:40.089097+0000","last_became_peered":"2024-03-21T12:51:40.089097+0000","last_unstale":"2024-03-21T12:52:17.051798+0000","last_undegraded":"2024-03-21T12:52:17.051798+0000","last_fullsized":"2024-03-21T12:52:17.051798+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:22:05.183786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157415+0000","last_change":"2024-03-21T12:51:42.092175+0000","last_active":"2024-03-21T12:52:17.157415+0000","last_peered":"2024-03-21T12:52:17.157415+0000","last_clean":"2024-03-21T12:52:17.157415+0000","last_became_active":"2024-03-21T12:51:42.091870+0000","last_became_peered":"2024-03-21T12:51:42.091870+0000","last_unstale":"2024-03-21T12:52:17.157415+0000","last_undegraded":"2024-03-21T12:52:17.157415+0000","last_fullsized":"2024-03-21T12:52:17.157415+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:51:22.230367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"69'8","reported_seq":53,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:51.132482+0000","last_change":"2024-03-21T12:51:40.092283+0000","last_active":"2024-03-21T12:52:51.132482+0000","last_peered":"2024-03-21T12:52:51.132482+0000","last_clean":"2024-03-21T12:52:51.132482+0000","last_became_active":"2024-03-21T12:51:40.092014+0000","last_became_peered":"2024-03-21T12:51:40.092014+0000","last_unstale":"2024-03-21T12:52:51.132482+0000","last_undegraded":"2024-03-21T12:52:51.132482+0000","last_fullsized":"2024-03-21T12:52:51.132482+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-23T00:37:19.924340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.049919+0000","last_change":"2024-03-21T12:51:36.073121+0000","last_active":"2024-03-21T12:52:17.049919+0000","last_peered":"2024-03-21T12:52:17.049919+0000","last_clean":"2024-03-21T12:52:17.049919+0000","last_became_active":"2024-03-21T12:51:36.072814+0000","last_became_peered":"2024-03-21T12:51:36.072814+0000","last_unstale":"2024-03-21T12:52:17.049919+0000","last_undegraded":"2024-03-21T12:52:17.049919+0000","last_fullsized":"2024-03-21T12:52:17.049919+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:52:54.557450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"69'15","reported_seq":54,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180995+0000","last_change":"2024-03-21T12:51:38.091725+0000","last_active":"2024-03-21T12:52:17.180995+0000","last_peered":"2024-03-21T12:52:17.180995+0000","last_clean":"2024-03-21T12:52:17.180995+0000","last_became_active":"2024-03-21T12:51:38.091209+0000","last_became_peered":"2024-03-21T12:51:38.091209+0000","last_unstale":"2024-03-21T12:52:17.180995+0000","last_undegraded":"2024-03-21T12:52:17.180995+0000","last_fullsized":"2024-03-21T12:52:17.180995+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:28:21.802719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.050384+0000","last_change":"2024-03-21T12:51:42.179420+0000","last_active":"2024-03-21T12:52:17.050384+0000","last_peered":"2024-03-21T12:52:17.050384+0000","last_clean":"2024-03-21T12:52:17.050384+0000","last_became_active":"2024-03-21T12:51:42.179010+0000","last_became_peered":"2024-03-21T12:51:42.179010+0000","last_unstale":"2024-03-21T12:52:17.050384+0000","last_undegraded":"2024-03-21T12:52:17.050384+0000","last_fullsized":"2024-03-21T12:52:17.050384+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:18:33.459751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.157893+0000","last_change":"2024-03-21T12:51:40.088111+0000","last_active":"2024-03-21T12:52:17.157893+0000","last_peered":"2024-03-21T12:52:17.157893+0000","last_clean":"2024-03-21T12:52:17.157893+0000","last_became_active":"2024-03-21T12:51:40.087818+0000","last_became_peered":"2024-03-21T12:51:40.087818+0000","last_unstale":"2024-03-21T12:52:17.157893+0000","last_undegraded":"2024-03-21T12:52:17.157893+0000","last_fullsized":"2024-03-21T12:52:17.157893+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:40:48.001709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051808+0000","last_change":"2024-03-21T12:51:36.077405+0000","last_active":"2024-03-21T12:52:17.051808+0000","last_peered":"2024-03-21T12:52:17.051808+0000","last_clean":"2024-03-21T12:52:17.051808+0000","last_became_active":"2024-03-21T12:51:36.077066+0000","last_became_peered":"2024-03-21T12:51:36.077066+0000","last_unstale":"2024-03-21T12:52:17.051808+0000","last_undegraded":"2024-03-21T12:52:17.051808+0000","last_fullsized":"2024-03-21T12:52:17.051808+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:50:48.083512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"69'11","reported_seq":48,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.054203+0000","last_change":"2024-03-21T12:51:38.088586+0000","last_active":"2024-03-21T12:52:17.054203+0000","last_peered":"2024-03-21T12:52:17.054203+0000","last_clean":"2024-03-21T12:52:17.054203+0000","last_became_active":"2024-03-21T12:51:38.087930+0000","last_became_peered":"2024-03-21T12:51:38.087930+0000","last_unstale":"2024-03-21T12:52:17.054203+0000","last_undegraded":"2024-03-21T12:52:17.054203+0000","last_fullsized":"2024-03-21T12:52:17.054203+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:25:58.539731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.049208+0000","last_change":"2024-03-21T12:51:42.095560+0000","last_active":"2024-03-21T12:52:17.049208+0000","last_peered":"2024-03-21T12:52:17.049208+0000","last_clean":"2024-03-21T12:52:17.049208+0000","last_became_active":"2024-03-21T12:51:42.092194+0000","last_became_peered":"2024-03-21T12:51:42.092194+0000","last_unstale":"2024-03-21T12:52:17.049208+0000","last_undegraded":"2024-03-21T12:52:17.049208+0000","last_fullsized":"2024-03-21T12:52:17.049208+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:18:32.865558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":25,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180374+0000","last_change":"2024-03-21T12:51:40.081988+0000","last_active":"2024-03-21T12:52:17.180374+0000","last_peered":"2024-03-21T12:52:17.180374+0000","last_clean":"2024-03-21T12:52:17.180374+0000","last_became_active":"2024-03-21T12:51:40.081670+0000","last_became_peered":"2024-03-21T12:51:40.081670+0000","last_unstale":"2024-03-21T12:52:17.180374+0000","last_undegraded":"2024-03-21T12:52:17.180374+0000","last_fullsized":"2024-03-21T12:52:17.180374+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T23:04:42.492985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.180031+0000","last_change":"2024-03-21T12:51:36.072552+0000","last_active":"2024-03-21T12:52:17.180031+0000","last_peered":"2024-03-21T12:52:17.180031+0000","last_clean":"2024-03-21T12:52:17.180031+0000","last_became_active":"2024-03-21T12:51:36.072254+0000","last_became_peered":"2024-03-21T12:51:36.072254+0000","last_unstale":"2024-03-21T12:52:17.180031+0000","last_undegraded":"2024-03-21T12:52:17.180031+0000","last_fullsized":"2024-03-21T12:52:17.180031+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:24:18.563715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"69'17","reported_seq":57,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.053143+0000","last_change":"2024-03-21T12:51:38.088641+0000","last_active":"2024-03-21T12:52:17.053143+0000","last_peered":"2024-03-21T12:52:17.053143+0000","last_clean":"2024-03-21T12:52:17.053143+0000","last_became_active":"2024-03-21T12:51:38.088079+0000","last_became_peered":"2024-03-21T12:51:38.088079+0000","last_unstale":"2024-03-21T12:52:17.053143+0000","last_undegraded":"2024-03-21T12:52:17.053143+0000","last_fullsized":"2024-03-21T12:52:17.053143+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:35:39.049497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":21,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.053702+0000","last_change":"2024-03-21T12:51:42.092898+0000","last_active":"2024-03-21T12:52:17.053702+0000","last_peered":"2024-03-21T12:52:17.053702+0000","last_clean":"2024-03-21T12:52:17.053702+0000","last_became_active":"2024-03-21T12:51:42.092540+0000","last_became_peered":"2024-03-21T12:51:42.092540+0000","last_unstale":"2024-03-21T12:52:17.053702+0000","last_undegraded":"2024-03-21T12:52:17.053702+0000","last_fullsized":"2024-03-21T12:52:17.053702+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:41.073401+0000","last_clean_scrub_stamp":"2024-03-21T12:51:41.073401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:09:35.569648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"69'8","reported_seq":50,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:51.132103+0000","last_change":"2024-03-21T12:51:40.093377+0000","last_active":"2024-03-21T12:52:51.132103+0000","last_peered":"2024-03-21T12:52:51.132103+0000","last_clean":"2024-03-21T12:52:51.132103+0000","last_became_active":"2024-03-21T12:51:40.093049+0000","last_became_peered":"2024-03-21T12:51:40.093049+0000","last_unstale":"2024-03-21T12:52:51.132103+0000","last_undegraded":"2024-03-21T12:52:51.132103+0000","last_fullsized":"2024-03-21T12:52:51.132103+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:39.070354+0000","last_clean_scrub_stamp":"2024-03-21T12:51:39.070354+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:53:00.969293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.051042+0000","last_change":"2024-03-21T12:51:36.080420+0000","last_active":"2024-03-21T12:52:17.051042+0000","last_peered":"2024-03-21T12:52:17.051042+0000","last_clean":"2024-03-21T12:52:17.051042+0000","last_became_active":"2024-03-21T12:51:36.080089+0000","last_became_peered":"2024-03-21T12:51:36.080089+0000","last_unstale":"2024-03-21T12:52:17.051042+0000","last_undegraded":"2024-03-21T12:52:17.051042+0000","last_fullsized":"2024-03-21T12:52:17.051042+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:35.053820+0000","last_clean_scrub_stamp":"2024-03-21T12:51:35.053820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T21:24:16.149768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"69'10","reported_seq":44,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-03-21T12:52:17.052226+0000","last_change":"2024-03-21T12:51:38.084752+0000","last_active":"2024-03-21T12:52:17.052226+0000","last_peered":"2024-03-21T12:52:17.052226+0000","last_clean":"2024-03-21T12:52:17.052226+0000","last_became_active":"2024-03-21T12:51:38.084371+0000","last_became_peered":"2024-03-21T12:51:38.084371+0000","last_unstale":"2024-03-21T12:52:17.052226+0000","last_undegraded":"2024-03-21T12:52:17.052226+0000","last_fullsized":"2024-03-21T12:52:17.052226+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T12:51:37.059815+0000","last_clean_scrub_stamp":"2024-03-21T12:51:37.059815+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:51:41.382057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":383,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1531,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201301,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28304,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716848,"statfs":{"total":95995035648,"available":95966052352,"internally_reserved":0,"allocated":1318912,"data_stored":874660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462937,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28296,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716856,"statfs":{"total":95995035648,"available":95966060544,"internally_reserved":0,"allocated":1306624,"data_stored":874532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27764,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717388,"statfs":{"total":95995035648,"available":95966605312,"internally_reserved":0,"allocated":761856,"data_stored":285131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738425,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32476,"kb_used_data":1300,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712676,"statfs":{"total":95995035648,"available":95961780224,"internally_reserved":0,"allocated":1331200,"data_stored":875326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411375,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31876,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713276,"statfs":{"total":95995035648,"available":95962394624,"internally_reserved":0,"allocated":712704,"data_stored":284011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149738,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27744,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717408,"statfs":{"total":95995035648,"available":95966625792,"internally_reserved":0,"allocated":741376,"data_stored":284131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986213,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27736,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717416,"statfs":{"total":95995035648,"available":95966633984,"internally_reserved":0,"allocated":733184,"data_stored":283845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724576,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31888,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713264,"statfs":{"total":95995035648,"available":95962382336,"internally_reserved":0,"allocated":724992,"data_stored":285036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T12:53:00.654 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-21T12:53:00.654 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-21T12:53:00.654 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-21T12:53:00.654 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753 shell --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 -- ceph health --format=json 2024-03-21T12:53:00.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:00 smithi155 ceph-mon[33846]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:00.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:00 smithi155 ceph-mon[33846]: from='client.14745 v1:172.21.15.43:0/2049333305' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T12:53:00.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:00 smithi043 ceph-mon[30062]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:00.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:00 smithi043 ceph-mon[30062]: from='client.14745 v1:172.21.15.43:0/2049333305' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T12:53:00.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:00 smithi043 ceph-mon[41699]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:00.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:00 smithi043 ceph-mon[41699]: from='client.14745 v1:172.21.15.43:0/2049333305' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T12:53:02.676 INFO:teuthology.orchestra.run.smithi043.stderr:Inferring config /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T12:53:02.864 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:02 smithi043 ceph-mon[30062]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:02.864 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:02 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:53:02.864 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:02 smithi043 ceph-mon[41699]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:02.864 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:02 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:53:02.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:02 smithi155 ceph-mon[33846]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:02.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:02 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:53:03.452 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:03 smithi043 systemd[1]: Starting Ceph alertmanager.a for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T12:53:04.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:04 smithi043 ceph-mon[30062]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:04.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:04 smithi043 ceph-mon[41699]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:04.735 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:04 smithi043 podman[79418]: 2024-03-21 12:53:04.390758985 +0000 UTC m=+0.019293748 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-03-21T12:53:04.735 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:04 smithi043 podman[79418]: 2024-03-21 12:53:04.492464243 +0000 UTC m=+0.120999046 volume create 0791bfc66574ee2b4d2aa30e46fc24ba3ccad44ead02c5f622c1b30eb07baba4 2024-03-21T12:53:04.735 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:04 smithi043 podman[79418]: 2024-03-21 12:53:04.592902526 +0000 UTC m=+0.221437325 container create 1b09ec7dc1eddce451c4420f8f2fc7af11f92066e0f64f933a2e37e7abaeae85 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-21T12:53:04.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:04 smithi155 ceph-mon[33846]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:05.146 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:04 smithi043 podman[79418]: 2024-03-21 12:53:04.895359414 +0000 UTC m=+0.523894172 container init 1b09ec7dc1eddce451c4420f8f2fc7af11f92066e0f64f933a2e37e7abaeae85 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-21T12:53:05.146 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:04 smithi043 podman[79418]: 2024-03-21 12:53:04.901384074 +0000 UTC m=+0.529918837 container start 1b09ec7dc1eddce451c4420f8f2fc7af11f92066e0f64f933a2e37e7abaeae85 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-21T12:53:05.146 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:04 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a[79449]: ts=2024-03-21T12:53:04.913Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-03-21T12:53:05.146 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:04 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a[79449]: ts=2024-03-21T12:53:04.914Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-03-21T12:53:05.146 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:04 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a[79449]: ts=2024-03-21T12:53:04.915Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.43 port=9094 2024-03-21T12:53:05.146 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:04 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a[79449]: ts=2024-03-21T12:53:04.916Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-03-21T12:53:05.146 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:04 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a[79449]: ts=2024-03-21T12:53:04.940Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-21T12:53:05.146 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:04 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a[79449]: ts=2024-03-21T12:53:04.940Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-21T12:53:05.146 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:04 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a[79449]: ts=2024-03-21T12:53:04.943Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-03-21T12:53:05.146 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:04 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a[79449]: ts=2024-03-21T12:53:04.943Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-03-21T12:53:05.146 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:04 smithi043 bash[79418]: 1b09ec7dc1eddce451c4420f8f2fc7af11f92066e0f64f933a2e37e7abaeae85 2024-03-21T12:53:05.147 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:04 smithi043 systemd[1]: Started Ceph alertmanager.a for c06c8d1a-e780-11ee-95cd-87774f69a715. 2024-03-21T12:53:05.147 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:05 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:53:05] ENGINE Bus STOPPING 2024-03-21T12:53:05.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.246+0000 7f1459eb4700 1 -- 172.21.15.43:0/1320362072 >> v1:172.21.15.43:6790/0 conn(0x7f145410aef0 legacy=0x7f145410d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:53:05.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.246+0000 7f1459eb4700 1 -- 172.21.15.43:0/1320362072 shutdown_connections 2024-03-21T12:53:05.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.246+0000 7f1459eb4700 1 -- 172.21.15.43:0/1320362072 >> 172.21.15.43:0/1320362072 conn(0x7f14540fc930 msgr2=0x7f14540fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:53:05.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.246+0000 7f1459eb4700 1 -- 172.21.15.43:0/1320362072 shutdown_connections 2024-03-21T12:53:05.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.246+0000 7f1459eb4700 1 -- 172.21.15.43:0/1320362072 wait complete. 2024-03-21T12:53:05.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.246+0000 7f1459eb4700 1 Processor -- start 2024-03-21T12:53:05.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.247+0000 7f1459eb4700 1 -- start start 2024-03-21T12:53:05.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.247+0000 7f1459eb4700 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f14541a9300 con 0x7f145410aef0 2024-03-21T12:53:05.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.247+0000 7f1459eb4700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f14541aa520 con 0x7f145410ea00 2024-03-21T12:53:05.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.247+0000 7f1459eb4700 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f14541ab760 con 0x7f14541030e0 2024-03-21T12:53:05.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.247+0000 7f14537fe700 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7f14541030e0 0x7f145410a620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:45920/0 (socket says 172.21.15.43:45920) 2024-03-21T12:53:05.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.247+0000 7f14537fe700 1 -- 172.21.15.43:0/679573390 learned_addr learned my addr 172.21.15.43:0/679573390 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T12:53:05.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.247+0000 7f1452ffd700 1 --1- 172.21.15.43:0/679573390 >> v1:172.21.15.43:6789/0 conn(0x7f145410aef0 0x7f14541a4210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52002/0 (socket says 172.21.15.43:52002) 2024-03-21T12:53:05.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.247+0000 7f1450ff9700 1 -- 172.21.15.43:0/679573390 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2758985628 0 0) 0x7f14541ab760 con 0x7f14541030e0 2024-03-21T12:53:05.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.247+0000 7f1450ff9700 1 -- 172.21.15.43:0/679573390 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14400035b0 con 0x7f14541030e0 2024-03-21T12:53:05.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.247+0000 7f1450ff9700 1 -- 172.21.15.43:0/679573390 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 830832745 0 0) 0x7f14541a9300 con 0x7f145410aef0 2024-03-21T12:53:05.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.247+0000 7f1450ff9700 1 -- 172.21.15.43:0/679573390 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14541ab760 con 0x7f145410aef0 2024-03-21T12:53:05.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.247+0000 7f1450ff9700 1 -- 172.21.15.43:0/679573390 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1594376181 0 0) 0x7f14541aa520 con 0x7f145410ea00 2024-03-21T12:53:05.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.247+0000 7f1450ff9700 1 -- 172.21.15.43:0/679573390 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14541a9300 con 0x7f145410ea00 2024-03-21T12:53:05.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.248+0000 7f1450ff9700 1 -- 172.21.15.43:0/679573390 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4241616580 0 0) 0x7f14541ab760 con 0x7f145410aef0 2024-03-21T12:53:05.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.248+0000 7f1450ff9700 1 -- 172.21.15.43:0/679573390 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f14541aa520 con 0x7f145410aef0 2024-03-21T12:53:05.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.248+0000 7f1450ff9700 1 -- 172.21.15.43:0/679573390 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f143c001430 con 0x7f145410aef0 2024-03-21T12:53:05.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.248+0000 7f1450ff9700 1 -- 172.21.15.43:0/679573390 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 127587525 0 0) 0x7f14400035b0 con 0x7f14541030e0 2024-03-21T12:53:05.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.248+0000 7f1450ff9700 1 -- 172.21.15.43:0/679573390 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f14541ab760 con 0x7f14541030e0 2024-03-21T12:53:05.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.248+0000 7f1450ff9700 1 -- 172.21.15.43:0/679573390 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3283401696 0 0) 0x7f14541a9300 con 0x7f145410ea00 2024-03-21T12:53:05.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.248+0000 7f1450ff9700 1 -- 172.21.15.43:0/679573390 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f14400035b0 con 0x7f145410ea00 2024-03-21T12:53:05.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.248+0000 7f1450ff9700 1 -- 172.21.15.43:0/679573390 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f1444001830 con 0x7f14541030e0 2024-03-21T12:53:05.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.248+0000 7f1450ff9700 1 -- 172.21.15.43:0/679573390 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f144c001bb0 con 0x7f145410ea00 2024-03-21T12:53:05.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.248+0000 7f1450ff9700 1 -- 172.21.15.43:0/679573390 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2624535691 0 0) 0x7f14541aa520 con 0x7f145410aef0 2024-03-21T12:53:05.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.248+0000 7f1450ff9700 1 -- 172.21.15.43:0/679573390 >> v1:172.21.15.43:6790/0 conn(0x7f14541030e0 legacy=0x7f145410a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:53:05.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.248+0000 7f1450ff9700 1 -- 172.21.15.43:0/679573390 >> v1:172.21.15.155:6789/0 conn(0x7f145410ea00 legacy=0x7f14541a79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:53:05.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.248+0000 7f1450ff9700 1 -- 172.21.15.43:0/679573390 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14541ac9a0 con 0x7f145410aef0 2024-03-21T12:53:05.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.248+0000 7f1459eb4700 1 -- 172.21.15.43:0/679573390 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14541a9530 con 0x7f145410aef0 2024-03-21T12:53:05.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.248+0000 7f1459eb4700 1 -- 172.21.15.43:0/679573390 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f14541a99f0 con 0x7f145410aef0 2024-03-21T12:53:05.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.248+0000 7f1450ff9700 1 -- 172.21.15.43:0/679573390 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f143c003680 con 0x7f145410aef0 2024-03-21T12:53:05.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.248+0000 7f1450ff9700 1 -- 172.21.15.43:0/679573390 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f143c004cd0 con 0x7f145410aef0 2024-03-21T12:53:05.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.249+0000 7f1459eb4700 1 -- 172.21.15.43:0/679573390 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1434005120 con 0x7f145410aef0 2024-03-21T12:53:05.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.249+0000 7f1450ff9700 1 -- 172.21.15.43:0/679573390 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7f143c004ef0 con 0x7f145410aef0 2024-03-21T12:53:05.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.250+0000 7f1450ff9700 1 -- 172.21.15.43:0/679573390 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(73..73 src has 1..73) v4 ==== 6259+0+0 (unknown 915400759 0 0) 0x7f143c089760 con 0x7f145410aef0 2024-03-21T12:53:05.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.254+0000 7f1450ff9700 1 -- 172.21.15.43:0/679573390 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f143c056d90 con 0x7f145410aef0 2024-03-21T12:53:05.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:05 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:53:05] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-21T12:53:05.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:05 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:53:05] ENGINE Bus STOPPED 2024-03-21T12:53:05.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:05 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:53:05] ENGINE Bus STARTING 2024-03-21T12:53:05.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:05 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:53:05] ENGINE Serving on http://:::9283 2024-03-21T12:53:05.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:05 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:53:05] ENGINE Bus STARTED 2024-03-21T12:53:05.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.507+0000 7f1459eb4700 1 -- 172.21.15.43:0/679573390 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1434005410 con 0x7f145410aef0 2024-03-21T12:53:05.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.507+0000 7f1450ff9700 1 -- 172.21.15.43:0/679573390 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (unknown 3730786198 0 4185958460) 0x7f143c05a7e0 con 0x7f145410aef0 2024-03-21T12:53:05.508 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-03-21T12:53:05.508 INFO:teuthology.orchestra.run.smithi043.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-21T12:53:05.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.509+0000 7f1459eb4700 1 -- 172.21.15.43:0/679573390 >> v1:172.21.15.43:6800/4199387216 conn(0x7f144006f290 legacy=0x7f1440071750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:53:05.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.509+0000 7f1459eb4700 1 -- 172.21.15.43:0/679573390 >> v1:172.21.15.43:6789/0 conn(0x7f145410aef0 legacy=0x7f14541a4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T12:53:05.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.509+0000 7f1459eb4700 1 -- 172.21.15.43:0/679573390 shutdown_connections 2024-03-21T12:53:05.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.509+0000 7f1459eb4700 1 -- 172.21.15.43:0/679573390 >> 172.21.15.43:0/679573390 conn(0x7f14540fc930 msgr2=0x7f14541025c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T12:53:05.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.509+0000 7f1459eb4700 1 -- 172.21.15.43:0/679573390 shutdown_connections 2024-03-21T12:53:05.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-03-21T12:53:05.509+0000 7f1459eb4700 1 -- 172.21.15.43:0/679573390 wait complete. 2024-03-21T12:53:06.226 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:06.226 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:06.226 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:06.227 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:06.227 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[30062]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:06.227 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[30062]: Regenerating cephadm self-signed grafana TLS certificates 2024-03-21T12:53:06.227 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:06.227 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:06.227 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-21T12:53:06.227 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-21T12:53:06.227 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:06.227 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[30062]: Deploying daemon grafana.a on smithi155 2024-03-21T12:53:06.227 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/679573390' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-21T12:53:06.227 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:06.227 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:06.228 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:06.228 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:06.228 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[41699]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:06.228 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[41699]: Regenerating cephadm self-signed grafana TLS certificates 2024-03-21T12:53:06.228 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:06.228 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:06.228 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-21T12:53:06.228 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-21T12:53:06.228 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:06.228 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[41699]: Deploying daemon grafana.a on smithi155 2024-03-21T12:53:06.228 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/679573390' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-21T12:53:06.253 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-21T12:53:06.253 INFO:tasks.cephadm:Setup complete, yielding 2024-03-21T12:53:06.253 INFO:teuthology.run_tasks:Running task workunit... 2024-03-21T12:53:06.270 INFO:tasks.workunit:Pulling workunits from ref 960fbbe4645c2becc8084669be194c95183c7753 2024-03-21T12:53:06.270 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-21T12:53:06.270 DEBUG:teuthology.orchestra.run.smithi043:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-21T12:53:06.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T12:53:06.286 INFO:teuthology.orchestra.run.smithi043.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-21T12:53:06.287 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-03-21T12:53:06.341 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-03-21T12:53:06.341 DEBUG:teuthology.orchestra.run.smithi043:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-03-21T12:53:06.396 INFO:tasks.workunit:timeout=3h 2024-03-21T12:53:06.396 INFO:tasks.workunit:cleanup=True 2024-03-21T12:53:06.396 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 960fbbe4645c2becc8084669be194c95183c7753 2024-03-21T12:53:06.452 INFO:tasks.workunit.client.0.smithi043.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-21T12:53:06.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:06 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:06.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:06 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:06.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:06 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:06.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:06 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:06.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:06 smithi155 ceph-mon[33846]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:06.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:06 smithi155 ceph-mon[33846]: Regenerating cephadm self-signed grafana TLS certificates 2024-03-21T12:53:06.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:06 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:06.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:06 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:06.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:06 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-21T12:53:06.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:06 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-21T12:53:06.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:06 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:06.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:06 smithi155 ceph-mon[33846]: Deploying daemon grafana.a on smithi155 2024-03-21T12:53:06.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:06 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/679573390' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-21T12:53:07.233 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:06 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a[79449]: ts=2024-03-21T12:53:06.916Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000090315s 2024-03-21T12:53:08.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:08 smithi155 ceph-mon[33846]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:08.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:08 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:08.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:53:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:53:08.473 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:08 smithi043 ceph-mon[30062]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:08.474 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:08 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:08.474 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:53:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:53:08.474 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:08 smithi043 ceph-mon[41699]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:08.474 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:08 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:09.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:09 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:53:09.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:09 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:53:09.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:09 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:53:10.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:10 smithi155 ceph-mon[33846]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:10.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:10 smithi043 ceph-mon[30062]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:10.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:10 smithi043 ceph-mon[41699]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:12.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:12 smithi155 ceph-mon[33846]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:12.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:12 smithi043 ceph-mon[30062]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:12.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:12 smithi043 ceph-mon[41699]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:14.415 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:14 smithi155 ceph-mon[33846]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:14.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:14 smithi043 ceph-mon[30062]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:14.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:14 smithi043 ceph-mon[41699]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:15.234 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:14 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a[79449]: ts=2024-03-21T12:53:14.918Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.00229588s 2024-03-21T12:53:15.368 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 systemd[1]: Starting Ceph grafana.a for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T12:53:15.704 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 podman[67371]: 2024-03-21 12:53:15.271488438 +0000 UTC m=+0.018227905 image pull 2bacad6d85d802cd176c146528adcc0e47a8a83e94bc4a8b7ba1a9b9ef2c65e7 quay.io/ceph/grafana:9.4.12 2024-03-21T12:53:15.704 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 podman[67371]: 2024-03-21 12:53:15.385387449 +0000 UTC m=+0.132126906 container create 2bd2d10258083709ad040a15e28c2aa77834b114275b013ecfdc9f429a1c2fb6 (image=quay.io/ceph/grafana:9.4.12, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a, maintainer=Grafana Labs ) 2024-03-21T12:53:15.704 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 podman[67371]: 2024-03-21 12:53:15.612720845 +0000 UTC m=+0.359460317 container init 2bd2d10258083709ad040a15e28c2aa77834b114275b013ecfdc9f429a1c2fb6 (image=quay.io/ceph/grafana:9.4.12, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a, maintainer=Grafana Labs ) 2024-03-21T12:53:15.704 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 podman[67371]: 2024-03-21 12:53:15.618384349 +0000 UTC m=+0.365123807 container start 2bd2d10258083709ad040a15e28c2aa77834b114275b013ecfdc9f429a1c2fb6 (image=quay.io/ceph/grafana:9.4.12, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a, maintainer=Grafana Labs ) 2024-03-21T12:53:15.704 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 bash[67371]: 2bd2d10258083709ad040a15e28c2aa77834b114275b013ecfdc9f429a1c2fb6 2024-03-21T12:53:15.954 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 systemd[1]: Started Ceph grafana.a for c06c8d1a-e780-11ee-95cd-87774f69a715. 2024-03-21T12:53:15.954 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=settings t=2024-03-21T12:53:15.766879515Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-03-21T12:53:15.955 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=settings t=2024-03-21T12:53:15.767057495Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-03-21T12:53:15.955 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=settings t=2024-03-21T12:53:15.767072692Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-03-21T12:53:15.955 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=settings t=2024-03-21T12:53:15.767077804Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-03-21T12:53:15.955 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=settings t=2024-03-21T12:53:15.767081765Z level=info msg="Config overridden from command line" arg="default.paths.data=/var/lib/grafana" 2024-03-21T12:53:15.955 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=settings t=2024-03-21T12:53:15.767085542Z level=info msg="Config overridden from command line" arg="default.paths.logs=/var/log/grafana" 2024-03-21T12:53:15.955 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=settings t=2024-03-21T12:53:15.76708974Z level=info msg="Config overridden from command line" arg="default.paths.plugins=/var/lib/grafana/plugins" 2024-03-21T12:53:15.955 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=settings t=2024-03-21T12:53:15.76709428Z level=info msg="Config overridden from command line" arg="default.paths.provisioning=/etc/grafana/provisioning" 2024-03-21T12:53:15.955 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=settings t=2024-03-21T12:53:15.767110568Z level=info msg="Config overridden from command line" arg="default.log.mode=console" 2024-03-21T12:53:15.955 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=settings t=2024-03-21T12:53:15.767116791Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-03-21T12:53:15.955 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=settings t=2024-03-21T12:53:15.767121658Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-03-21T12:53:15.955 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=settings t=2024-03-21T12:53:15.767125604Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-03-21T12:53:15.955 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=settings t=2024-03-21T12:53:15.767129901Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-03-21T12:53:15.955 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=settings t=2024-03-21T12:53:15.76713412Z level=info msg="Path Home" path=/usr/share/grafana 2024-03-21T12:53:15.956 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=settings t=2024-03-21T12:53:15.76713837Z level=info msg="Path Data" path=/var/lib/grafana 2024-03-21T12:53:15.956 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=settings t=2024-03-21T12:53:15.767142393Z level=info msg="Path Logs" path=/var/log/grafana 2024-03-21T12:53:15.956 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=settings t=2024-03-21T12:53:15.767146601Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-03-21T12:53:15.956 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=settings t=2024-03-21T12:53:15.767152037Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-03-21T12:53:15.956 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=settings t=2024-03-21T12:53:15.767156212Z level=info msg="App mode production" 2024-03-21T12:53:15.956 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=sqlstore t=2024-03-21T12:53:15.767215609Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-03-21T12:53:15.956 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=sqlstore t=2024-03-21T12:53:15.767237508Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-03-21T12:53:15.956 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.768500047Z level=info msg="Starting DB migrations" 2024-03-21T12:53:15.956 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.768586709Z level=info msg="Executing migration" id="create migration_log table" 2024-03-21T12:53:15.956 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.769133672Z level=info msg="Executing migration" id="create user table" 2024-03-21T12:53:15.956 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.769584219Z level=info msg="Executing migration" id="add unique index user.login" 2024-03-21T12:53:15.956 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.770037053Z level=info msg="Executing migration" id="add unique index user.email" 2024-03-21T12:53:15.956 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.770476656Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-03-21T12:53:15.956 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.77105339Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-03-21T12:53:15.956 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.771429898Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-03-21T12:53:15.957 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.772177362Z level=info msg="Executing migration" id="create user table v2" 2024-03-21T12:53:15.957 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.772557671Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-03-21T12:53:15.957 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.772957323Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-03-21T12:53:15.957 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.77336796Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-03-21T12:53:15.957 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.77375571Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-03-21T12:53:15.957 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.774137556Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-03-21T12:53:15.957 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.77463858Z level=info msg="Executing migration" id="Update user table charset" 2024-03-21T12:53:15.957 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.774952988Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-03-21T12:53:15.958 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.775360947Z level=info msg="Executing migration" id="Add missing user data" 2024-03-21T12:53:15.958 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.77592071Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-03-21T12:53:15.959 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.776323042Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-03-21T12:53:15.959 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.776760713Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-03-21T12:53:15.959 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.777197611Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-03-21T12:53:15.959 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.778562849Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-03-21T12:53:15.959 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.779031735Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-03-21T12:53:15.959 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.77947682Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-03-21T12:53:15.959 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.779854617Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-03-21T12:53:15.959 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.780227025Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-03-21T12:53:15.959 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.780607535Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-03-21T12:53:15.959 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.780889669Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-03-21T12:53:15.959 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.781252964Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-03-21T12:53:15.959 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.781635765Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-03-21T12:53:15.959 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.7820005Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-03-21T12:53:15.960 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.782333792Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-03-21T12:53:15.960 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.783120184Z level=info msg="Executing migration" id="create temp_user v2" 2024-03-21T12:53:15.960 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.783486761Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-03-21T12:53:15.960 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.783874514Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-03-21T12:53:15.960 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.78422436Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-03-21T12:53:15.960 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.784606212Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-03-21T12:53:15.960 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.784983207Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-03-21T12:53:15.960 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.785274174Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-03-21T12:53:15.960 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.785610697Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-03-21T12:53:15.960 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.785937899Z level=info msg="Executing migration" id="create star table" 2024-03-21T12:53:15.960 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.786267186Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-03-21T12:53:15.960 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.786670343Z level=info msg="Executing migration" id="create org table v1" 2024-03-21T12:53:15.960 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.787033278Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-03-21T12:53:15.960 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.787405562Z level=info msg="Executing migration" id="create org_user table v1" 2024-03-21T12:53:15.960 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.787771983Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-03-21T12:53:15.961 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.788202776Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-03-21T12:53:15.961 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.789366148Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-03-21T12:53:15.961 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.789883951Z level=info msg="Executing migration" id="Update org table charset" 2024-03-21T12:53:15.961 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.790737506Z level=info msg="Executing migration" id="Update org_user table charset" 2024-03-21T12:53:15.961 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.791558776Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-03-21T12:53:15.961 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.792358556Z level=info msg="Executing migration" id="create dashboard table" 2024-03-21T12:53:15.961 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.795306388Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-03-21T12:53:15.961 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.798167056Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-03-21T12:53:15.961 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.801015421Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-03-21T12:53:15.961 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.801457024Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-03-21T12:53:15.961 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.804398115Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-03-21T12:53:15.961 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.807199487Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-03-21T12:53:15.961 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.810101976Z level=info msg="Executing migration" id="create dashboard v2" 2024-03-21T12:53:15.961 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.810532042Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-03-21T12:53:15.961 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.813527471Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-03-21T12:53:15.962 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.816464337Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-03-21T12:53:15.962 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.819276901Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-03-21T12:53:15.962 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.819794291Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-03-21T12:53:15.962 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.822624458Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-03-21T12:53:15.962 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.825595183Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-03-21T12:53:15.962 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.828417528Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-03-21T12:53:15.962 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.829006839Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-03-21T12:53:15.962 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.831812164Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-03-21T12:53:15.962 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.83239805Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-03-21T12:53:15.962 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.834569318Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-03-21T12:53:15.962 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.837278937Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-03-21T12:53:15.962 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.837616438Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-03-21T12:53:15.962 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.838524567Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-03-21T12:53:15.962 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.83936744Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-03-21T12:53:15.963 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.840347791Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-03-21T12:53:15.963 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.841070188Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-03-21T12:53:15.963 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.841908728Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-03-21T12:53:15.963 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.842871516Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-03-21T12:53:15.963 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.843606702Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-03-21T12:53:15.963 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.844029151Z level=info msg="Executing migration" id="Update dashboard title length" 2024-03-21T12:53:15.963 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.846944806Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-03-21T12:53:15.963 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.84748156Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-03-21T12:53:15.963 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.849723478Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-03-21T12:53:15.963 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.85270904Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-03-21T12:53:15.963 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.853225606Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-03-21T12:53:15.963 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.856212031Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-03-21T12:53:15.963 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.858998077Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-03-21T12:53:15.963 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.8619718Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-03-21T12:53:15.964 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.862490145Z level=info msg="Executing migration" id="Add check_sum column" 2024-03-21T12:53:15.964 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.86549809Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-03-21T12:53:15.964 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.868308961Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-03-21T12:53:15.964 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.871127794Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-03-21T12:53:15.964 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.871481946Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-03-21T12:53:15.964 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.874376591Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-03-21T12:53:15.964 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.877094691Z level=info msg="Executing migration" id="create data_source table" 2024-03-21T12:53:15.964 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.880122441Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-03-21T12:53:15.964 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.880726455Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-03-21T12:53:15.964 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.883677354Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-03-21T12:53:15.964 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.886370247Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-03-21T12:53:15.964 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.889034688Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-03-21T12:53:15.964 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.890429121Z level=info msg="Executing migration" id="create data_source table v2" 2024-03-21T12:53:15.965 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.893171997Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-03-21T12:53:15.965 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.895756424Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-03-21T12:53:15.965 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.898335498Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-03-21T12:53:15.965 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.901067763Z level=info msg="Executing migration" id="Add column with_credentials" 2024-03-21T12:53:15.965 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.903568962Z level=info msg="Executing migration" id="Add secure json data column" 2024-03-21T12:53:15.965 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.906195575Z level=info msg="Executing migration" id="Update data_source table charset" 2024-03-21T12:53:15.965 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.908817978Z level=info msg="Executing migration" id="Update initial version to 1" 2024-03-21T12:53:15.965 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.909150813Z level=info msg="Executing migration" id="Add read_only data column" 2024-03-21T12:53:15.965 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.911938842Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-03-21T12:53:15.965 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.914646155Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-03-21T12:53:15.965 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.917042564Z level=info msg="Executing migration" id="Add uid column" 2024-03-21T12:53:15.965 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.917627167Z level=info msg="Executing migration" id="Update uid value" 2024-03-21T12:53:15.965 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.920391662Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-03-21T12:53:15.965 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.922942584Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-03-21T12:53:15.965 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.923616856Z level=info msg="Executing migration" id="create api_key table" 2024-03-21T12:53:15.966 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.924032101Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-03-21T12:53:15.966 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.92484961Z level=info msg="Executing migration" id="add index api_key.key" 2024-03-21T12:53:15.966 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.925697106Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-03-21T12:53:15.966 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.926551649Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-03-21T12:53:15.966 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.926968408Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-03-21T12:53:15.966 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.927756829Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-03-21T12:53:15.966 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.928214747Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-03-21T12:53:15.966 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.92952945Z level=info msg="Executing migration" id="create api_key table v2" 2024-03-21T12:53:15.966 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.930452469Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-03-21T12:53:15.966 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.931320478Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-03-21T12:53:15.966 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.932015708Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-03-21T12:53:15.966 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.933013782Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-03-21T12:53:15.966 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.933648562Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-03-21T12:53:15.966 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.934509429Z level=info msg="Executing migration" id="Update api_key table charset" 2024-03-21T12:53:15.966 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.935420467Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-03-21T12:53:15.967 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.936112063Z level=info msg="Executing migration" id="Add service account foreign key" 2024-03-21T12:53:15.967 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.936976303Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-03-21T12:53:15.967 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.937950564Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-03-21T12:53:15.967 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.938700761Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-03-21T12:53:15.967 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.939709576Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-03-21T12:53:15.967 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.940431278Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-03-21T12:53:15.967 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.941381727Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-03-21T12:53:15.967 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.942128562Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-03-21T12:53:15.967 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.943027674Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-03-21T12:53:15.967 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.943875977Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-03-21T12:53:15.967 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.944831251Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-03-21T12:53:15.967 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.945551773Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-03-21T12:53:15.967 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.946432678Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-03-21T12:53:15.967 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.947312931Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-03-21T12:53:15.968 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.948129811Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-03-21T12:53:15.968 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.949087411Z level=info msg="Executing migration" id="create quota table v1" 2024-03-21T12:53:15.968 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.949830286Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-03-21T12:53:15.968 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.950675636Z level=info msg="Executing migration" id="Update quota table charset" 2024-03-21T12:53:15.968 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.951525643Z level=info msg="Executing migration" id="create plugin_setting table" 2024-03-21T12:53:15.968 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.954607155Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-03-21T12:53:15.968 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.95727987Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-03-21T12:53:16.205 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-mon[33846]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:16.206 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:16.206 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:16.206 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:16.206 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:16.209 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.960058474Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-03-21T12:53:16.209 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.962979456Z level=info msg="Executing migration" id="create session table" 2024-03-21T12:53:16.209 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.965773476Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-03-21T12:53:16.209 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.968680087Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-03-21T12:53:16.209 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.971386468Z level=info msg="Executing migration" id="create playlist table v2" 2024-03-21T12:53:16.210 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.974082377Z level=info msg="Executing migration" id="create playlist item table v2" 2024-03-21T12:53:16.210 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.976964353Z level=info msg="Executing migration" id="Update playlist table charset" 2024-03-21T12:53:16.210 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.979692725Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-03-21T12:53:16.210 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.982674223Z level=info msg="Executing migration" id="drop preferences table v2" 2024-03-21T12:53:16.210 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.985408676Z level=info msg="Executing migration" id="drop preferences table v3" 2024-03-21T12:53:16.211 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.9880749Z level=info msg="Executing migration" id="create preferences table v3" 2024-03-21T12:53:16.211 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.990956709Z level=info msg="Executing migration" id="Update preferences table charset" 2024-03-21T12:53:16.211 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.993605114Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-03-21T12:53:16.211 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.996402528Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-03-21T12:53:16.211 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:15 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:15.999374583Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-03-21T12:53:16.211 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.002047274Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-03-21T12:53:16.212 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.004734812Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-03-21T12:53:16.212 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.007540129Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-03-21T12:53:16.212 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.010297557Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-03-21T12:53:16.212 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.013131488Z level=info msg="Executing migration" id="create alert table v1" 2024-03-21T12:53:16.212 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.015657737Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-03-21T12:53:16.212 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.018359524Z level=info msg="Executing migration" id="add index alert state" 2024-03-21T12:53:16.212 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.0212589Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-03-21T12:53:16.213 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.023821849Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-03-21T12:53:16.213 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.026259459Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-03-21T12:53:16.213 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.029058706Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-03-21T12:53:16.213 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.031828061Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-03-21T12:53:16.213 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.034382426Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-03-21T12:53:16.213 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.036994675Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-03-21T12:53:16.213 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.039673054Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-03-21T12:53:16.214 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.042414249Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-03-21T12:53:16.214 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.045345244Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-03-21T12:53:16.214 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.04794976Z level=info msg="Executing migration" id="Add column is_default" 2024-03-21T12:53:16.214 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.048821915Z level=info msg="Executing migration" id="Add column frequency" 2024-03-21T12:53:16.214 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.049665773Z level=info msg="Executing migration" id="Add column send_reminder" 2024-03-21T12:53:16.214 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.050518258Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-03-21T12:53:16.215 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.051371428Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-03-21T12:53:16.215 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.052187412Z level=info msg="Executing migration" id="Update alert table charset" 2024-03-21T12:53:16.215 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.053115324Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-03-21T12:53:16.215 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.053832974Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-03-21T12:53:16.215 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.054787307Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-03-21T12:53:16.215 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.055612865Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-03-21T12:53:16.215 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.056351755Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-03-21T12:53:16.216 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.057324861Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-03-21T12:53:16.216 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.058037569Z level=info msg="Executing migration" id="Add for to alert table" 2024-03-21T12:53:16.216 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.058860453Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-03-21T12:53:16.216 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.059613708Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-03-21T12:53:16.216 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.060528232Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-03-21T12:53:16.216 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.06141068Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-03-21T12:53:16.216 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.063806507Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-03-21T12:53:16.216 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.066642133Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-03-21T12:53:16.217 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.069295047Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-03-21T12:53:16.217 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.072095802Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-03-21T12:53:16.217 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.074924522Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-03-21T12:53:16.217 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.077626432Z level=info msg="Executing migration" id="create annotation table v5" 2024-03-21T12:53:16.217 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.08032398Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-03-21T12:53:16.217 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.083108753Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-03-21T12:53:16.217 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.086055703Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-03-21T12:53:16.218 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.088623465Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-03-21T12:53:16.218 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.09142983Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-03-21T12:53:16.218 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.094176087Z level=info msg="Executing migration" id="Update annotation table charset" 2024-03-21T12:53:16.218 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.096831205Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-03-21T12:53:16.218 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.09953234Z level=info msg="Executing migration" id="Drop category_id index" 2024-03-21T12:53:16.218 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.102023166Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-03-21T12:53:16.219 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.104809748Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-03-21T12:53:16.219 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.107576557Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-03-21T12:53:16.219 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.110209397Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-03-21T12:53:16.219 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.112948256Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-03-21T12:53:16.219 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.115564576Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-03-21T12:53:16.219 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.118225024Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-03-21T12:53:16.220 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.12112141Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-03-21T12:53:16.220 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.123573793Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-03-21T12:53:16.220 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.126316627Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-03-21T12:53:16.220 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.127142024Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-03-21T12:53:16.220 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.128190251Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-03-21T12:53:16.220 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.129013016Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-03-21T12:53:16.220 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.129846752Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-03-21T12:53:16.221 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.130647201Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-03-21T12:53:16.221 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.131609228Z level=info msg="Executing migration" id="Add epoch_end column" 2024-03-21T12:53:16.221 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.132536458Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-03-21T12:53:16.221 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.133327784Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-03-21T12:53:16.221 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.1342373Z level=info msg="Executing migration" id="Move region to single row" 2024-03-21T12:53:16.221 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.134952599Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-03-21T12:53:16.221 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.135842921Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-03-21T12:53:16.221 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.136608975Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-03-21T12:53:16.222 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.137449313Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-03-21T12:53:16.222 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.138351203Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-03-21T12:53:16.222 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.139090237Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-03-21T12:53:16.222 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.13997422Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-03-21T12:53:16.222 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.140766113Z level=info msg="Executing migration" id="create test_data table" 2024-03-21T12:53:16.222 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.141607321Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-03-21T12:53:16.222 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.142072696Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-03-21T12:53:16.222 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.142961189Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-03-21T12:53:16.223 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.143831665Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-03-21T12:53:16.223 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.144780688Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-03-21T12:53:16.223 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.145581227Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-03-21T12:53:16.223 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.146413442Z level=info msg="Executing migration" id="create team table" 2024-03-21T12:53:16.223 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.147392353Z level=info msg="Executing migration" id="add index team.org_id" 2024-03-21T12:53:16.223 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.148131745Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-03-21T12:53:16.223 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.148943239Z level=info msg="Executing migration" id="create team member table" 2024-03-21T12:53:16.223 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.149832666Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-03-21T12:53:16.224 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.150694069Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-03-21T12:53:16.224 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.151571489Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-03-21T12:53:16.224 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.152431464Z level=info msg="Executing migration" id="Add column email to team table" 2024-03-21T12:53:16.224 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.153327216Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-03-21T12:53:16.224 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.154168828Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-03-21T12:53:16.224 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.155029485Z level=info msg="Executing migration" id="create dashboard acl table" 2024-03-21T12:53:16.224 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.155891156Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-03-21T12:53:16.225 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.156668658Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-03-21T12:53:16.225 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.159586072Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-03-21T12:53:16.225 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.160112755Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-03-21T12:53:16.225 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.162943943Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-03-21T12:53:16.225 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.165760462Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-03-21T12:53:16.225 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.166232352Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-03-21T12:53:16.225 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.16668465Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-03-21T12:53:16.226 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.167097391Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-03-21T12:53:16.226 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.167401076Z level=info msg="Executing migration" id="create tag table" 2024-03-21T12:53:16.226 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.167823106Z level=info msg="Executing migration" id="add index tag.key_value" 2024-03-21T12:53:16.226 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.168265598Z level=info msg="Executing migration" id="create login attempt table" 2024-03-21T12:53:16.226 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.168716039Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-03-21T12:53:16.226 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.169169403Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-03-21T12:53:16.227 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.169635663Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-03-21T12:53:16.227 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.172032456Z level=info msg="Executing migration" id="create login_attempt v2" 2024-03-21T12:53:16.227 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.172445669Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-03-21T12:53:16.227 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.172892518Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-03-21T12:53:16.227 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.173227835Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-03-21T12:53:16.227 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.173640992Z level=info msg="Executing migration" id="create user auth table" 2024-03-21T12:53:16.228 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.174038585Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-03-21T12:53:16.228 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.174501454Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-03-21T12:53:16.228 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.174813799Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-03-21T12:53:16.228 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.175687691Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-03-21T12:53:16.228 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.176534409Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-03-21T12:53:16.228 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.177506618Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-03-21T12:53:16.228 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.178387413Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-03-21T12:53:16.228 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.178850048Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-03-21T12:53:16.228 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.179730312Z level=info msg="Executing migration" id="create server_lock table" 2024-03-21T12:53:16.228 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.180135176Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-03-21T12:53:16.228 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.180605997Z level=info msg="Executing migration" id="create user auth token table" 2024-03-21T12:53:16.228 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.181072151Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-03-21T12:53:16.228 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.183517509Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-03-21T12:53:16.229 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.18400625Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-03-21T12:53:16.229 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.184476261Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-03-21T12:53:16.229 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.185388972Z level=info msg="Executing migration" id="create cache_data table" 2024-03-21T12:53:16.229 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.185834936Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-03-21T12:53:16.229 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.186292764Z level=info msg="Executing migration" id="create short_url table v1" 2024-03-21T12:53:16.229 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.186755096Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-03-21T12:53:16.229 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.18938836Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-03-21T12:53:16.229 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.189730554Z level=info msg="Executing migration" id="delete alert_definition table" 2024-03-21T12:53:16.229 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.190011602Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-03-21T12:53:16.229 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.190494252Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-03-21T12:53:16.229 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.19096097Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-03-21T12:53:16.229 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.191401411Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-03-21T12:53:16.229 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.191707606Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-03-21T12:53:16.229 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.192135882Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-03-21T12:53:16.230 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.192536804Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-03-21T12:53:16.230 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.193024879Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-03-21T12:53:16.230 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.193479162Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-03-21T12:53:16.230 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.194372701Z level=info msg="Executing migration" id="drop alert_definition table" 2024-03-21T12:53:16.230 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.194840457Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-03-21T12:53:16.230 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.195184139Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-03-21T12:53:16.230 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.19769196Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-03-21T12:53:16.230 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.20066273Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-03-21T12:53:16.230 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.203212163Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-03-21T12:53:16.230 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.203591345Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-03-21T12:53:16.230 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.205979755Z level=info msg="Executing migration" id="create alert_instance table" 2024-03-21T12:53:16.230 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.208771872Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-03-21T12:53:16.457 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.211531021Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-03-21T12:53:16.457 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.212099615Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-03-21T12:53:16.457 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.213406465Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-03-21T12:53:16.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.214256774Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-03-21T12:53:16.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.217130445Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-03-21T12:53:16.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.221974748Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-03-21T12:53:16.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.226377591Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-03-21T12:53:16.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.22726869Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-03-21T12:53:16.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.228194516Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-03-21T12:53:16.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.229208664Z level=info msg="Executing migration" id="create alert_rule table" 2024-03-21T12:53:16.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.229653197Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-03-21T12:53:16.459 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.230484597Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-03-21T12:53:16.459 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.231274165Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-03-21T12:53:16.459 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.232117694Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-03-21T12:53:16.459 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.232408269Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-03-21T12:53:16.459 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.233369215Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-03-21T12:53:16.459 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.234280217Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-03-21T12:53:16.461 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.235205214Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-03-21T12:53:16.461 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.235634746Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-03-21T12:53:16.461 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.23646678Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-03-21T12:53:16.461 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.237377226Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-03-21T12:53:16.461 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.238293846Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-03-21T12:53:16.462 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.238739375Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-03-21T12:53:16.462 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.239687416Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-03-21T12:53:16.462 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.240618938Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-03-21T12:53:16.462 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.2414744Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-03-21T12:53:16.462 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.241917463Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-03-21T12:53:16.462 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.242768543Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-03-21T12:53:16.463 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.243716788Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-03-21T12:53:16.463 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.244620844Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-03-21T12:53:16.463 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.24568633Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-03-21T12:53:16.463 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.246648094Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-03-21T12:53:16.463 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.247557371Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-03-21T12:53:16.464 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.248493432Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-03-21T12:53:16.464 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.249428826Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-03-21T12:53:16.464 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.250407073Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-03-21T12:53:16.464 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.25110395Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-03-21T12:53:16.464 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.252061064Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-03-21T12:53:16.464 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.252354668Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-03-21T12:53:16.464 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.25333031Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-03-21T12:53:16.465 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.254300336Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-03-21T12:53:16.465 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.255284909Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-03-21T12:53:16.465 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.255702538Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-03-21T12:53:16.465 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.258522382Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-03-21T12:53:16.465 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.261402146Z level=info msg="Executing migration" id="create provenance_type table" 2024-03-21T12:53:16.465 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.264173601Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-03-21T12:53:16.465 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.264806615Z level=info msg="Executing migration" id="create alert_image table" 2024-03-21T12:53:16.466 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.267558865Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-03-21T12:53:16.466 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.270347952Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-03-21T12:53:16.466 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.273145423Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-03-21T12:53:16.466 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.276067449Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-03-21T12:53:16.466 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.278692238Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-03-21T12:53:16.466 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.278763328Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-03-21T12:53:16.466 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.281658755Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-03-21T12:53:16.466 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.284417927Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-03-21T12:53:16.467 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.285019188Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-03-21T12:53:16.467 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.285127839Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-03-21T12:53:16.467 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.285236579Z level=info msg="alerts found to migrate" alerts=0 2024-03-21T12:53:16.467 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.287701252Z level=info msg="Executing migration" id="create library_element table v1" 2024-03-21T12:53:16.467 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.290600982Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-03-21T12:53:16.468 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.29338906Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-03-21T12:53:16.468 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.293867116Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-03-21T12:53:16.468 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.296663165Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-03-21T12:53:16.469 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.299497299Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-03-21T12:53:16.469 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.302067626Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-03-21T12:53:16.469 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.302431464Z level=info msg="Executing migration" id="create data_keys table" 2024-03-21T12:53:16.469 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.30535237Z level=info msg="Executing migration" id="create secrets table" 2024-03-21T12:53:16.469 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.307902721Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-03-21T12:53:16.469 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.313479079Z level=info msg="Executing migration" id="add name column into data_keys" 2024-03-21T12:53:16.469 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.314513618Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-03-21T12:53:16.469 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.317282997Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-03-21T12:53:16.469 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.322957515Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-03-21T12:53:16.469 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.32878858Z level=info msg="Executing migration" id="create kv_store table v1" 2024-03-21T12:53:16.469 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.329194091Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-03-21T12:53:16.469 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.332061644Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-03-21T12:53:16.469 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.334658051Z level=info msg="Executing migration" id="create permission table" 2024-03-21T12:53:16.469 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.337200362Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-03-21T12:53:16.470 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.337649251Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-03-21T12:53:16.470 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.340476637Z level=info msg="Executing migration" id="create role table" 2024-03-21T12:53:16.470 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.343309083Z level=info msg="Executing migration" id="add column display_name" 2024-03-21T12:53:16.470 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.344473187Z level=info msg="Executing migration" id="add column group_name" 2024-03-21T12:53:16.470 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.345496615Z level=info msg="Executing migration" id="add index role.org_id" 2024-03-21T12:53:16.470 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.346336951Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-03-21T12:53:16.470 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.347335713Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-03-21T12:53:16.470 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.347961436Z level=info msg="Executing migration" id="create team role table" 2024-03-21T12:53:16.470 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.348375291Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-03-21T12:53:16.470 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.349171747Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-03-21T12:53:16.470 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.349954408Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-03-21T12:53:16.470 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.350838264Z level=info msg="Executing migration" id="create user role table" 2024-03-21T12:53:16.470 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.351634566Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-03-21T12:53:16.470 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.352410988Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-03-21T12:53:16.471 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.353344748Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-03-21T12:53:16.471 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.354074766Z level=info msg="Executing migration" id="create builtin role table" 2024-03-21T12:53:16.471 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.354489409Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-03-21T12:53:16.471 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.355423583Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-03-21T12:53:16.471 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.356070431Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-03-21T12:53:16.471 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.357179503Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-03-21T12:53:16.471 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.357707223Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-03-21T12:53:16.471 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.358531539Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-03-21T12:53:16.471 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.359528536Z level=info msg="Executing migration" id="add unique index role.uid" 2024-03-21T12:53:16.471 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.360237212Z level=info msg="Executing migration" id="create seed assignment table" 2024-03-21T12:53:16.471 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.360628354Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-03-21T12:53:16.471 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.361517099Z level=info msg="Executing migration" id="add column hidden to role table" 2024-03-21T12:53:16.472 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.362660948Z level=info msg="Executing migration" id="create query_history table v1" 2024-03-21T12:53:16.472 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.363681984Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-03-21T12:53:16.472 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.364237676Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-03-21T12:53:16.472 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.365129378Z level=info msg="Executing migration" id="teams permissions migration" 2024-03-21T12:53:16.472 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.365771703Z level=info msg="Executing migration" id="dashboard permissions" 2024-03-21T12:53:16.472 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.366215658Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-03-21T12:53:16.472 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.367038692Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-03-21T12:53:16.473 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.367360735Z level=info msg="Executing migration" id="alerting notification permissions" 2024-03-21T12:53:16.473 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.367749013Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-03-21T12:53:16.473 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.368687691Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-03-21T12:53:16.473 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.369396627Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-03-21T12:53:16.474 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.370533658Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-03-21T12:53:16.474 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.370879336Z level=info msg="Executing migration" id="create correlation table v1" 2024-03-21T12:53:16.474 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.373845637Z level=info msg="Executing migration" id="add index correlations.uid" 2024-03-21T12:53:16.474 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.376581932Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-03-21T12:53:16.474 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.379619252Z level=info msg="Executing migration" id="add correlation config column" 2024-03-21T12:53:16.474 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.380958824Z level=info msg="Executing migration" id="create entity_events table" 2024-03-21T12:53:16.474 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.383784475Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-03-21T12:53:16.474 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.386538256Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-03-21T12:53:16.474 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.386627336Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-03-21T12:53:16.474 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.389203209Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-03-21T12:53:16.474 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.389308103Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-03-21T12:53:16.474 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.389599507Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-03-21T12:53:16.474 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.392428387Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-03-21T12:53:16.474 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.395131544Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-03-21T12:53:16.475 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.398059798Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-03-21T12:53:16.475 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.39855902Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-03-21T12:53:16.475 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.401328352Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-03-21T12:53:16.475 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.404163753Z level=info msg="Executing migration" id="Drop public config table" 2024-03-21T12:53:16.475 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.406989731Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-03-21T12:53:16.475 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.407610145Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-03-21T12:53:16.475 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.410495844Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-03-21T12:53:16.475 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.413076174Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-03-21T12:53:16.475 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.415765974Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-03-21T12:53:16.475 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.419624368Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-03-21T12:53:16.475 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.422409569Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-03-21T12:53:16.475 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.425040854Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-03-21T12:53:16.475 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.427735247Z level=info msg="Executing migration" id="add share column" 2024-03-21T12:53:16.475 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.42895076Z level=info msg="Executing migration" id="create default alerting folders" 2024-03-21T12:53:16.476 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.429751769Z level=info msg="Executing migration" id="create file table" 2024-03-21T12:53:16.476 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.430566488Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-03-21T12:53:16.476 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.43138646Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-03-21T12:53:16.476 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.431901529Z level=info msg="Executing migration" id="create file_meta table" 2024-03-21T12:53:16.476 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.432303902Z level=info msg="Executing migration" id="file table idx: path key" 2024-03-21T12:53:16.476 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.433162163Z level=info msg="Executing migration" id="set path collation in file table" 2024-03-21T12:53:16.476 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.434008826Z level=info msg="Executing migration" id="managed permissions migration" 2024-03-21T12:53:16.476 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.434545483Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-03-21T12:53:16.476 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.435424876Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-03-21T12:53:16.476 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.436412555Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-03-21T12:53:16.476 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.437617242Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-03-21T12:53:16.476 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.438416701Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-03-21T12:53:16.476 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.439387767Z level=info msg="Executing migration" id="update group index for alert rules" 2024-03-21T12:53:16.476 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.44011144Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-03-21T12:53:16.477 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.440941736Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-03-21T12:53:16.477 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.441545294Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-03-21T12:53:16.477 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.442716394Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-03-21T12:53:16.477 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.443933172Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-03-21T12:53:16.477 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.444786316Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-03-21T12:53:16.477 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.457605147Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-03-21T12:53:16.477 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.458186905Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-03-21T12:53:16.477 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.459045749Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-03-21T12:53:16.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:16 smithi043 ceph-mon[30062]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:16.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:16 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:16.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:16 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:16.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:16 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:16.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:16 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:16.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:16 smithi043 ceph-mon[41699]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:16.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:16 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:16.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:16 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:16.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:16 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:16.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:16 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:16.707 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.465486324Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-03-21T12:53:16.708 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.466237311Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-03-21T12:53:16.708 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.466561634Z level=info msg="Executing migration" id="create folder table" 2024-03-21T12:53:16.708 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.466965753Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-03-21T12:53:16.708 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.46939574Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-03-21T12:53:16.708 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.471968706Z level=info msg="Executing migration" id="Update folder title length" 2024-03-21T12:53:16.708 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.474667142Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-03-21T12:53:16.708 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=migrator t=2024-03-21T12:53:16.477349387Z level=info msg="migrations completed" performed=485 skipped=0 duration=708.788548ms 2024-03-21T12:53:16.708 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=sqlstore t=2024-03-21T12:53:16.477765109Z level=info msg="Created default organization" 2024-03-21T12:53:16.708 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=plugin.finder t=2024-03-21T12:53:16.511958722Z level=warn msg="Skipping finding plugins as directory does not exist" path=/usr/share/grafana/plugins-bundled 2024-03-21T12:53:16.709 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=secrets t=2024-03-21T12:53:16.512059074Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-03-21T12:53:16.709 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=query_data t=2024-03-21T12:53:16.514081992Z level=info msg="Query Service initialization" 2024-03-21T12:53:16.709 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=live.push_http t=2024-03-21T12:53:16.51679637Z level=info msg="Live Push Gateway initialization" 2024-03-21T12:53:16.977 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:16 smithi043 systemd[1]: Starting Ceph node-exporter.a for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T12:53:17.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:17 smithi043 ceph-mon[30062]: Deploying daemon node-exporter.a on smithi043 2024-03-21T12:53:17.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:17 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:17.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:17 smithi043 ceph-mon[41699]: Deploying daemon node-exporter.a on smithi043 2024-03-21T12:53:17.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:17 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:17.234 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:16 smithi043 bash[79857]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-03-21T12:53:17.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:17 smithi155 ceph-mon[33846]: Deploying daemon node-exporter.a on smithi043 2024-03-21T12:53:17.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:17 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:17.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:17 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=infra.usagestats.collector t=2024-03-21T12:53:17.122966726Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-03-21T12:53:17.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:17 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=provisioning.datasources t=2024-03-21T12:53:17.123971806Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-03-21T12:53:17.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:17 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=provisioning.datasources t=2024-03-21T12:53:17.134913832Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-03-21T12:53:17.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:17 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=provisioning.alerting t=2024-03-21T12:53:17.145662864Z level=info msg="starting to provision alerting" 2024-03-21T12:53:17.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:17 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=provisioning.alerting t=2024-03-21T12:53:17.145674449Z level=info msg="finished to provision alerting" 2024-03-21T12:53:17.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:17 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=ngalert.state.manager t=2024-03-21T12:53:17.14583813Z level=info msg="Warming state cache for startup" 2024-03-21T12:53:17.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:17 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=grafanaStorageLogger t=2024-03-21T12:53:17.145860391Z level=info msg="storage starting" 2024-03-21T12:53:17.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:17 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=ngalert.state.manager t=2024-03-21T12:53:17.145964172Z level=info msg="State cache has been initialized" states=0 duration=124.134µs 2024-03-21T12:53:17.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:17 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=ngalert.multiorg.alertmanager t=2024-03-21T12:53:17.146203434Z level=info msg="starting MultiOrg Alertmanager" 2024-03-21T12:53:17.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:17 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=ticker t=2024-03-21T12:53:17.146223936Z level=info msg=starting first_tick=2024-03-21T12:53:20Z 2024-03-21T12:53:17.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:17 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=http.server t=2024-03-21T12:53:17.148058827Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-03-21T12:53:17.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:17 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=sqlstore.transactions t=2024-03-21T12:53:17.156245274Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-03-21T12:53:17.459 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:17 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=sqlstore.transactions t=2024-03-21T12:53:17.173120966Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-03-21T12:53:17.459 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:17 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=sqlstore.transactions t=2024-03-21T12:53:17.173115161Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-03-21T12:53:17.459 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:17 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=sqlstore.transactions t=2024-03-21T12:53:17.185243097Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=2 code="database is locked" 2024-03-21T12:53:17.459 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:17 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=sqlstore.transactions t=2024-03-21T12:53:17.195393572Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=3 code="database is locked" 2024-03-21T12:53:17.459 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:17 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=sqlstore.transactions t=2024-03-21T12:53:17.205606037Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=4 code="database is locked" 2024-03-21T12:53:17.459 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 12:53:17 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=auth t=2024-03-21T12:53:17.208874408Z level=error msg="failed to lock and execute cleanup of expired auth token" error="database is locked" 2024-03-21T12:53:17.733 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:17 smithi043 bash[79857]: Getting image source signatures 2024-03-21T12:53:17.734 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:17 smithi043 bash[79857]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-03-21T12:53:17.734 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:17 smithi043 bash[79857]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-03-21T12:53:17.734 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:17 smithi043 bash[79857]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-03-21T12:53:18.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:18 smithi043 ceph-mon[30062]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:18.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:18 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:53:18.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:53:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:53:18.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:18 smithi043 ceph-mon[41699]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:18.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:18 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:53:18.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:53:18 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:53:18.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:18 smithi155 ceph-mon[33846]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:18.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:18 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:53:19.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:19 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:53:19.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:53:19.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:53:19.484 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 bash[79857]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-03-21T12:53:19.484 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 bash[79857]: Writing manifest to image destination 2024-03-21T12:53:19.762 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 podman[79857]: 2024-03-21 12:53:19.437246578 +0000 UTC m=+2.556462953 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-03-21T12:53:19.762 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 podman[79857]: 2024-03-21 12:53:19.53564572 +0000 UTC m=+2.654862075 container create 314670e24b1af1ab3f986d4913c9318da0945d2f062404fc5d19f1cf64bf471b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-21T12:53:20.164 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 podman[79857]: 2024-03-21 12:53:19.762849142 +0000 UTC m=+2.882065498 container init 314670e24b1af1ab3f986d4913c9318da0945d2f062404fc5d19f1cf64bf471b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-21T12:53:20.165 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 podman[79857]: 2024-03-21 12:53:19.768596162 +0000 UTC m=+2.887812518 container start 314670e24b1af1ab3f986d4913c9318da0945d2f062404fc5d19f1cf64bf471b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-21T12:53:20.165 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.772Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-03-21T12:53:20.165 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.772Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-03-21T12:53:20.165 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-03-21T12:53:20.165 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-03-21T12:53:20.165 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-03-21T12:53:20.165 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-03-21T12:53:20.165 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-03-21T12:53:20.165 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=arp 2024-03-21T12:53:20.165 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=bcache 2024-03-21T12:53:20.165 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=bonding 2024-03-21T12:53:20.166 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=btrfs 2024-03-21T12:53:20.166 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=conntrack 2024-03-21T12:53:20.166 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=cpu 2024-03-21T12:53:20.166 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-03-21T12:53:20.166 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=diskstats 2024-03-21T12:53:20.166 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=dmi 2024-03-21T12:53:20.166 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=edac 2024-03-21T12:53:20.167 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=entropy 2024-03-21T12:53:20.167 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-03-21T12:53:20.167 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=filefd 2024-03-21T12:53:20.167 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=filesystem 2024-03-21T12:53:20.167 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=hwmon 2024-03-21T12:53:20.167 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=infiniband 2024-03-21T12:53:20.167 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=ipvs 2024-03-21T12:53:20.167 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=loadavg 2024-03-21T12:53:20.167 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=mdadm 2024-03-21T12:53:20.167 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=meminfo 2024-03-21T12:53:20.167 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=netclass 2024-03-21T12:53:20.167 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=netdev 2024-03-21T12:53:20.167 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=netstat 2024-03-21T12:53:20.167 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=nfs 2024-03-21T12:53:20.167 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=nfsd 2024-03-21T12:53:20.168 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=nvme 2024-03-21T12:53:20.168 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=os 2024-03-21T12:53:20.168 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-03-21T12:53:20.168 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.773Z caller=node_exporter.go:117 level=info collector=pressure 2024-03-21T12:53:20.168 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.774Z caller=node_exporter.go:117 level=info collector=rapl 2024-03-21T12:53:20.168 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.774Z caller=node_exporter.go:117 level=info collector=schedstat 2024-03-21T12:53:20.168 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.774Z caller=node_exporter.go:117 level=info collector=selinux 2024-03-21T12:53:20.168 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.774Z caller=node_exporter.go:117 level=info collector=sockstat 2024-03-21T12:53:20.168 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.774Z caller=node_exporter.go:117 level=info collector=softnet 2024-03-21T12:53:20.168 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.774Z caller=node_exporter.go:117 level=info collector=stat 2024-03-21T12:53:20.168 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.774Z caller=node_exporter.go:117 level=info collector=tapestats 2024-03-21T12:53:20.168 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.774Z caller=node_exporter.go:117 level=info collector=textfile 2024-03-21T12:53:20.168 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.774Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-03-21T12:53:20.168 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.774Z caller=node_exporter.go:117 level=info collector=time 2024-03-21T12:53:20.168 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.774Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-03-21T12:53:20.168 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.774Z caller=node_exporter.go:117 level=info collector=uname 2024-03-21T12:53:20.169 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.774Z caller=node_exporter.go:117 level=info collector=vmstat 2024-03-21T12:53:20.169 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.774Z caller=node_exporter.go:117 level=info collector=xfs 2024-03-21T12:53:20.169 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.774Z caller=node_exporter.go:117 level=info collector=zfs 2024-03-21T12:53:20.169 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.774Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-03-21T12:53:20.169 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a[79907]: ts=2024-03-21T12:53:19.774Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-03-21T12:53:20.169 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 bash[79857]: 314670e24b1af1ab3f986d4913c9318da0945d2f062404fc5d19f1cf64bf471b 2024-03-21T12:53:20.169 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 12:53:19 smithi043 systemd[1]: Started Ceph node-exporter.a for c06c8d1a-e780-11ee-95cd-87774f69a715. 2024-03-21T12:53:20.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:20 smithi155 ceph-mon[33846]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:20.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:20 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:20.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:20 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:20.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:20 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:20.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:20 smithi043 ceph-mon[30062]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:20.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:20 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:20.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:20 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:20.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:20 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:20.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:20 smithi043 ceph-mon[41699]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:20.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:20 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:20.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:20 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:20.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:20 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:21.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:21 smithi155 ceph-mon[33846]: Deploying daemon node-exporter.b on smithi155 2024-03-21T12:53:21.457 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:21 smithi155 bash[67660]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-03-21T12:53:21.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:21 smithi043 ceph-mon[30062]: Deploying daemon node-exporter.b on smithi155 2024-03-21T12:53:21.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:21 smithi043 ceph-mon[41699]: Deploying daemon node-exporter.b on smithi155 2024-03-21T12:53:21.957 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:21 smithi155 bash[67660]: Getting image source signatures 2024-03-21T12:53:21.957 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:21 smithi155 bash[67660]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-03-21T12:53:21.957 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:21 smithi155 bash[67660]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-03-21T12:53:21.957 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:21 smithi155 bash[67660]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-03-21T12:53:22.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:22 smithi155 ceph-mon[33846]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:22.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:22 smithi043 ceph-mon[30062]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:22.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:22 smithi043 ceph-mon[41699]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:23.403 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:22 smithi155 bash[67660]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-03-21T12:53:23.403 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 bash[67660]: Writing manifest to image destination 2024-03-21T12:53:23.690 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 podman[67660]: 2024-03-21 12:53:23.390747957 +0000 UTC m=+2.310607547 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-03-21T12:53:23.690 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 podman[67660]: 2024-03-21 12:53:23.480976939 +0000 UTC m=+2.400836528 container create 390703071fd0d5186f2107ca75772f2722883635a64b59d187b35e9df5f6ca17 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-21T12:53:23.957 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 podman[67660]: 2024-03-21 12:53:23.69091273 +0000 UTC m=+2.610772334 container init 390703071fd0d5186f2107ca75772f2722883635a64b59d187b35e9df5f6ca17 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-21T12:53:23.957 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 podman[67660]: 2024-03-21 12:53:23.696240833 +0000 UTC m=+2.616100431 container start 390703071fd0d5186f2107ca75772f2722883635a64b59d187b35e9df5f6ca17 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-21T12:53:23.957 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.700Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-03-21T12:53:23.958 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.700Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-03-21T12:53:23.958 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.700Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-03-21T12:53:23.958 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.700Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-03-21T12:53:23.958 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.700Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-03-21T12:53:23.958 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-03-21T12:53:23.958 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-03-21T12:53:23.958 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=arp 2024-03-21T12:53:23.958 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=bcache 2024-03-21T12:53:23.958 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=bonding 2024-03-21T12:53:23.958 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=btrfs 2024-03-21T12:53:23.958 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=conntrack 2024-03-21T12:53:23.958 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=cpu 2024-03-21T12:53:23.958 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-03-21T12:53:23.959 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=diskstats 2024-03-21T12:53:23.959 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=dmi 2024-03-21T12:53:23.959 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=edac 2024-03-21T12:53:23.959 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=entropy 2024-03-21T12:53:23.959 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-03-21T12:53:23.959 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=filefd 2024-03-21T12:53:23.959 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=filesystem 2024-03-21T12:53:23.959 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=hwmon 2024-03-21T12:53:23.960 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=infiniband 2024-03-21T12:53:23.960 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=ipvs 2024-03-21T12:53:23.960 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=loadavg 2024-03-21T12:53:23.960 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=mdadm 2024-03-21T12:53:23.960 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=meminfo 2024-03-21T12:53:23.960 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=netclass 2024-03-21T12:53:23.960 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=netdev 2024-03-21T12:53:23.960 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=netstat 2024-03-21T12:53:23.960 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=nfs 2024-03-21T12:53:23.960 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=nfsd 2024-03-21T12:53:23.960 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=nvme 2024-03-21T12:53:23.960 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=os 2024-03-21T12:53:23.961 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-03-21T12:53:23.961 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=pressure 2024-03-21T12:53:23.961 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=rapl 2024-03-21T12:53:23.961 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=schedstat 2024-03-21T12:53:23.961 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=selinux 2024-03-21T12:53:23.961 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=sockstat 2024-03-21T12:53:23.961 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=softnet 2024-03-21T12:53:23.961 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=stat 2024-03-21T12:53:23.961 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=tapestats 2024-03-21T12:53:23.961 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=textfile 2024-03-21T12:53:23.961 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-03-21T12:53:23.961 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=time 2024-03-21T12:53:23.961 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-03-21T12:53:23.961 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=uname 2024-03-21T12:53:23.962 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=vmstat 2024-03-21T12:53:23.962 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=xfs 2024-03-21T12:53:23.962 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=node_exporter.go:117 level=info collector=zfs 2024-03-21T12:53:23.962 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-03-21T12:53:23.962 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b[67710]: ts=2024-03-21T12:53:23.701Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-03-21T12:53:23.962 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 bash[67660]: 390703071fd0d5186f2107ca75772f2722883635a64b59d187b35e9df5f6ca17 2024-03-21T12:53:23.962 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 12:53:23 smithi155 systemd[1]: Started Ceph node-exporter.b for c06c8d1a-e780-11ee-95cd-87774f69a715. 2024-03-21T12:53:24.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:24 smithi155 ceph-mon[33846]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:24.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:24 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:24.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:24 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:24.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:24 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:24.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:24 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:24.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:24 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:24.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:24 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:53:24.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:24 smithi043 ceph-mon[30062]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:24.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:24 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:24.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:24 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:24.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:24 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:24.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:24 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:24.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:24 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:24.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:24 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:53:24.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:24 smithi043 ceph-mon[41699]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:24.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:24 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:24.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:24 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:24.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:24 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:24.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:24 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:24.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:24 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:24.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:24 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:53:26.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:26 smithi155 ceph-mon[33846]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:26.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:26 smithi043 ceph-mon[30062]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:26.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:26 smithi043 ceph-mon[41699]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:28.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:53:28 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:53:28.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:28 smithi155 ceph-mon[33846]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:28.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:28 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:28.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:28 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:28.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:28 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:28.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:53:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:53:28.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:28 smithi043 ceph-mon[30062]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:28.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:28 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:28.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:28 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:28.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:28 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:28.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:28 smithi043 ceph-mon[41699]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:28.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:28 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:28.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:28 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:28.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:28 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:29.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:29 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:53:29.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:29 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:53:29.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:29 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:53:30.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:30 smithi155 ceph-mon[33846]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:30.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:30 smithi043 ceph-mon[30062]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:30.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:30 smithi043 ceph-mon[41699]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:32.380 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:32 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:32.381 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:32 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:32.381 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:32 smithi043 ceph-mon[30062]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:32.381 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:32 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:53:32.381 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:32 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:53:32.381 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:32 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:32.381 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:32 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:32.381 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:32 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:32.381 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:32 smithi043 ceph-mon[41699]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:32.382 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:32 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:53:32.382 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:32 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:53:32.382 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:32 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:32.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:32 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:32.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:32 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:32.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:32 smithi155 ceph-mon[33846]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:32.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:32 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:53:32.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:32 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:53:32.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:32 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:33.324 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:33 smithi043 ceph-mon[30062]: Reconfiguring alertmanager.a (dependencies changed)... 2024-03-21T12:53:33.324 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:33 smithi043 ceph-mon[30062]: Reconfiguring daemon alertmanager.a on smithi043 2024-03-21T12:53:33.324 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:33 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:53:33.324 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:33 smithi043 ceph-mon[41699]: Reconfiguring alertmanager.a (dependencies changed)... 2024-03-21T12:53:33.324 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:33 smithi043 ceph-mon[41699]: Reconfiguring daemon alertmanager.a on smithi043 2024-03-21T12:53:33.325 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:33 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:53:33.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:33 smithi155 ceph-mon[33846]: Reconfiguring alertmanager.a (dependencies changed)... 2024-03-21T12:53:33.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:33 smithi155 ceph-mon[33846]: Reconfiguring daemon alertmanager.a on smithi043 2024-03-21T12:53:33.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:33 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:53:34.359 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:34 smithi155 ceph-mon[33846]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:34.363 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:34 smithi043 ceph-mon[30062]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:34.363 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:34 smithi043 ceph-mon[41699]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:35.473 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:35 smithi043 systemd[1]: Stopping Ceph alertmanager.a for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T12:53:35.733 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:35 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a[79449]: ts=2024-03-21T12:53:35.473Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-03-21T12:53:35.734 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:35 smithi043 podman[80720]: 2024-03-21 12:53:35.504420338 +0000 UTC m=+0.138071129 container died 1b09ec7dc1eddce451c4420f8f2fc7af11f92066e0f64f933a2e37e7abaeae85 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-21T12:53:36.132 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:35 smithi043 podman[80720]: 2024-03-21 12:53:35.912688279 +0000 UTC m=+0.546339061 container cleanup 1b09ec7dc1eddce451c4420f8f2fc7af11f92066e0f64f933a2e37e7abaeae85 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-21T12:53:36.132 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:35 smithi043 bash[80720]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a 2024-03-21T12:53:36.382 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:36 smithi043 ceph-mon[30062]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:36.382 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:36 smithi043 ceph-mon[41699]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:36.382 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:36 smithi043 podman[80739]: 2024-03-21 12:53:36.160174823 +0000 UTC m=+0.651722964 container remove 1b09ec7dc1eddce451c4420f8f2fc7af11f92066e0f64f933a2e37e7abaeae85 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-21T12:53:36.382 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:36 smithi043 podman[80739]: 2024-03-21 12:53:36.239745254 +0000 UTC m=+0.731293352 volume remove 0791bfc66574ee2b4d2aa30e46fc24ba3ccad44ead02c5f622c1b30eb07baba4 2024-03-21T12:53:36.383 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:36 smithi043 systemd[1]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-03-21T12:53:36.383 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:36 smithi043 systemd[1]: Stopped Ceph alertmanager.a for c06c8d1a-e780-11ee-95cd-87774f69a715. 2024-03-21T12:53:36.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:36 smithi155 ceph-mon[33846]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:36.657 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:36 smithi043 systemd[1]: Starting Ceph alertmanager.a for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T12:53:36.657 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:36 smithi043 podman[80837]: 2024-03-21 12:53:36.590443208 +0000 UTC m=+0.114690433 volume create 6d396cca714dfe6c9bf7bee0c28d0c194b9bdca4e6e936c5e77ffda2e34c5a54 2024-03-21T12:53:36.657 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:36 smithi043 podman[80837]: 2024-03-21 12:53:36.493286948 +0000 UTC m=+0.017534148 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-03-21T12:53:36.951 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:36 smithi043 podman[80837]: 2024-03-21 12:53:36.657540393 +0000 UTC m=+0.181787635 container create 1687430f4e4f00222d1dba9d45cdfa5f19df49c7ebb57625bbbb2da9313ad43a (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-21T12:53:37.234 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:36 smithi043 podman[80837]: 2024-03-21 12:53:36.951978891 +0000 UTC m=+0.476226094 container init 1687430f4e4f00222d1dba9d45cdfa5f19df49c7ebb57625bbbb2da9313ad43a (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-21T12:53:37.234 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:36 smithi043 podman[80837]: 2024-03-21 12:53:36.957668764 +0000 UTC m=+0.481915947 container start 1687430f4e4f00222d1dba9d45cdfa5f19df49c7ebb57625bbbb2da9313ad43a (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-21T12:53:37.234 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:36 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a[80851]: ts=2024-03-21T12:53:36.970Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-03-21T12:53:37.234 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:36 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a[80851]: ts=2024-03-21T12:53:36.970Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-03-21T12:53:37.234 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:36 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a[80851]: ts=2024-03-21T12:53:36.971Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.43 port=9094 2024-03-21T12:53:37.234 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:36 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a[80851]: ts=2024-03-21T12:53:36.971Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-03-21T12:53:37.234 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:36 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a[80851]: ts=2024-03-21T12:53:36.995Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-21T12:53:37.234 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:36 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a[80851]: ts=2024-03-21T12:53:36.995Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-21T12:53:37.234 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:36 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a[80851]: ts=2024-03-21T12:53:36.998Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-03-21T12:53:37.235 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:36 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a[80851]: ts=2024-03-21T12:53:36.998Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-03-21T12:53:37.235 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:37 smithi043 bash[80837]: 1687430f4e4f00222d1dba9d45cdfa5f19df49c7ebb57625bbbb2da9313ad43a 2024-03-21T12:53:37.235 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:37 smithi043 systemd[1]: Started Ceph alertmanager.a for c06c8d1a-e780-11ee-95cd-87774f69a715. 2024-03-21T12:53:38.209 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:53:38 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:53:38.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:38 smithi155 ceph-mon[33846]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:38.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:38 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:38.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:38 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:38.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:38 smithi155 ceph-mon[33846]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-21T12:53:38.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:38 smithi155 ceph-mon[33846]: Reconfiguring daemon prometheus.a on smithi155 2024-03-21T12:53:38.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:53:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:53:38.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:38 smithi043 ceph-mon[30062]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:38.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:38 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:38.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:38 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:38.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:38 smithi043 ceph-mon[30062]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-21T12:53:38.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:38 smithi043 ceph-mon[30062]: Reconfiguring daemon prometheus.a on smithi155 2024-03-21T12:53:38.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:38 smithi043 ceph-mon[41699]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:38.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:38 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:38.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:38 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:38.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:38 smithi043 ceph-mon[41699]: Reconfiguring prometheus.a (dependencies changed)... 2024-03-21T12:53:38.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:38 smithi043 ceph-mon[41699]: Reconfiguring daemon prometheus.a on smithi155 2024-03-21T12:53:39.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:39 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:53:39.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:39 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:53:39.234 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a[80851]: ts=2024-03-21T12:53:38.972Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000500599s 2024-03-21T12:53:39.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:39 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:53:40.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:40 smithi155 ceph-mon[33846]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:40.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:40 smithi043 ceph-mon[30062]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:40.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:40 smithi043 ceph-mon[41699]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:40.939 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:40 smithi155 systemd[1]: Stopping Ceph prometheus.a for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T12:53:41.207 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:40 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:53:40.939Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-03-21T12:53:41.207 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:40 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:53:40.939Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-03-21T12:53:41.207 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:40 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:53:40.939Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-03-21T12:53:41.207 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:40 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:53:40.939Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-03-21T12:53:41.207 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:40 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:53:40.939Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-03-21T12:53:41.207 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:40 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:53:40.939Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-03-21T12:53:41.208 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:40 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:53:40.940Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-03-21T12:53:41.208 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:40 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:53:40.940Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-03-21T12:53:41.208 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:40 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:53:40.940Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-03-21T12:53:41.208 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:40 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:53:40.940Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-03-21T12:53:41.208 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:40 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:53:40.940Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-03-21T12:53:41.208 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:40 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[65304]: ts=2024-03-21T12:53:40.940Z caller=main.go:1147 level=info msg="See you next time!" 2024-03-21T12:53:41.208 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:40 smithi155 podman[68752]: 2024-03-21 12:53:40.950670923 +0000 UTC m=+0.121003481 container died ee988b220acfa4567606aaea4308fdf464a7fa6e963c6bcbed4b460b6a676c92 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-21T12:53:41.529 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:41 smithi155 podman[68752]: 2024-03-21 12:53:41.336741805 +0000 UTC m=+0.507074366 container cleanup ee988b220acfa4567606aaea4308fdf464a7fa6e963c6bcbed4b460b6a676c92 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-21T12:53:41.529 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:41 smithi155 bash[68752]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a 2024-03-21T12:53:41.955 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:41 smithi155 podman[68771]: 2024-03-21 12:53:41.529706776 +0000 UTC m=+0.577688551 container remove ee988b220acfa4567606aaea4308fdf464a7fa6e963c6bcbed4b460b6a676c92 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-21T12:53:41.955 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:41 smithi155 systemd[1]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@prometheus.a.service: Deactivated successfully. 2024-03-21T12:53:41.955 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:41 smithi155 systemd[1]: Stopped Ceph prometheus.a for c06c8d1a-e780-11ee-95cd-87774f69a715. 2024-03-21T12:53:41.955 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:41 smithi155 systemd[1]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@prometheus.a.service: Consumed 1.177s CPU time. 2024-03-21T12:53:41.955 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:41 smithi155 systemd[1]: Starting Ceph prometheus.a for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T12:53:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:42 smithi155 ceph-mon[33846]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:42.207 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:41 smithi155 podman[68870]: 2024-03-21 12:53:41.859276472 +0000 UTC m=+0.018956073 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-03-21T12:53:42.207 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:41 smithi155 podman[68870]: 2024-03-21 12:53:41.974279954 +0000 UTC m=+0.133959582 container create a2db772e1052bf870709ffcaf6442b7fbd95c0f0a78ebfdbd121732889047fb4 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-21T12:53:42.420 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:42 smithi043 ceph-mon[30062]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:42.420 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:42 smithi043 ceph-mon[41699]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:42.708 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:42 smithi155 podman[68870]: 2024-03-21 12:53:42.250362068 +0000 UTC m=+0.410041669 container init a2db772e1052bf870709ffcaf6442b7fbd95c0f0a78ebfdbd121732889047fb4 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-21T12:53:42.708 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:42 smithi155 podman[68870]: 2024-03-21 12:53:42.255568073 +0000 UTC m=+0.415247655 container start a2db772e1052bf870709ffcaf6442b7fbd95c0f0a78ebfdbd121732889047fb4 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-21T12:53:42.708 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T12:53:42.279Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-03-21T12:53:42.708 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T12:53:42.279Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-03-21T12:53:42.708 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T12:53:42.279Z caller=main.go:570 level=info host_details="(Linux 5.14.0-430.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Mar 14 17:54:49 UTC 2024 x86_64 smithi155 (none))" 2024-03-21T12:53:42.708 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T12:53:42.279Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-03-21T12:53:42.708 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T12:53:42.279Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-03-21T12:53:42.708 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T12:53:42.281Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-03-21T12:53:42.708 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T12:53:42.281Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-03-21T12:53:42.708 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T12:53:42.282Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-03-21T12:53:42.708 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T12:53:42.282Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-03-21T12:53:42.709 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T12:53:42.284Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-03-21T12:53:42.709 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T12:53:42.284Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.964µs 2024-03-21T12:53:42.709 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T12:53:42.284Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-03-21T12:53:42.709 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T12:53:42.285Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-03-21T12:53:42.709 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T12:53:42.285Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-03-21T12:53:42.709 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T12:53:42.285Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=18.448µs wal_replay_duration=573.693µs wbl_replay_duration=106ns total_replay_duration=610.391µs 2024-03-21T12:53:42.709 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T12:53:42.286Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-03-21T12:53:42.709 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T12:53:42.286Z caller=main.go:1029 level=info msg="TSDB started" 2024-03-21T12:53:42.709 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T12:53:42.286Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-03-21T12:53:42.709 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T12:53:42.305Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=19.467116ms db_storage=769ns remote_storage=1.44µs web_handler=335ns query_engine=561ns scrape=194.513µs scrape_sd=85.445µs notify=49.682µs notify_sd=22.442µs rules=18.822691ms tracing=5.549µs 2024-03-21T12:53:42.709 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T12:53:42.305Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-03-21T12:53:42.709 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T12:53:42.305Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-03-21T12:53:42.709 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:42 smithi155 bash[68870]: a2db772e1052bf870709ffcaf6442b7fbd95c0f0a78ebfdbd121732889047fb4 2024-03-21T12:53:42.709 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 12:53:42 smithi155 systemd[1]: Started Ceph prometheus.a for c06c8d1a-e780-11ee-95cd-87774f69a715. 2024-03-21T12:53:42.734 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:42 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:53:42] ENGINE Bus STOPPING 2024-03-21T12:53:42.734 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:42 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:53:42] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-21T12:53:42.734 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:42 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:53:42] ENGINE Bus STOPPED 2024-03-21T12:53:42.734 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:42 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:53:42] ENGINE Bus STARTING 2024-03-21T12:53:42.734 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:42 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:53:42] ENGINE Serving on http://:::9283 2024-03-21T12:53:42.734 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:42 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:53:42] ENGINE Bus STARTED 2024-03-21T12:53:42.734 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:42 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:53:42] ENGINE Bus STOPPING 2024-03-21T12:53:42.734 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:42 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:53:42] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-21T12:53:42.735 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:42 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:53:42] ENGINE Bus STOPPED 2024-03-21T12:53:42.735 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:42 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:53:42] ENGINE Bus STARTING 2024-03-21T12:53:42.735 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:42 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:53:42] ENGINE Serving on http://:::9283 2024-03-21T12:53:42.735 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:42 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:53:42] ENGINE Bus STARTED 2024-03-21T12:53:42.735 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:42 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:53:42] ENGINE Bus STOPPING 2024-03-21T12:53:42.735 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:42 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:53:42] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-21T12:53:42.735 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:42 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:53:42] ENGINE Bus STOPPED 2024-03-21T12:53:42.735 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:42 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:53:42] ENGINE Bus STARTING 2024-03-21T12:53:43.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:42 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:53:42] ENGINE Serving on http://:::9283 2024-03-21T12:53:43.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:42 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:12:53:42] ENGINE Bus STARTED 2024-03-21T12:53:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:43 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:43 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:43 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-21T12:53:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:43 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-21T12:53:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:43 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"}]: dispatch 2024-03-21T12:53:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:43 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"}]: dispatch 2024-03-21T12:53:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:43 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:43 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T12:53:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:43 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T12:53:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:43 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi155.front.sepia.ceph.com:9095"}]: dispatch 2024-03-21T12:53:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:43 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi155.front.sepia.ceph.com:9095"}]: dispatch 2024-03-21T12:53:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:43 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:43 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-21T12:53:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:43 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-21T12:53:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:43 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi155.front.sepia.ceph.com:3000"}]: dispatch 2024-03-21T12:53:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:43 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi155.front.sepia.ceph.com:3000"}]: dispatch 2024-03-21T12:53:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:43 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:43 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:53:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-21T12:53:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-21T12:53:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"}]: dispatch 2024-03-21T12:53:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"}]: dispatch 2024-03-21T12:53:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T12:53:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T12:53:43.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi155.front.sepia.ceph.com:9095"}]: dispatch 2024-03-21T12:53:43.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi155.front.sepia.ceph.com:9095"}]: dispatch 2024-03-21T12:53:43.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:43.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-21T12:53:43.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-21T12:53:43.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi155.front.sepia.ceph.com:3000"}]: dispatch 2024-03-21T12:53:43.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi155.front.sepia.ceph.com:3000"}]: dispatch 2024-03-21T12:53:43.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:43.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:53:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-21T12:53:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-21T12:53:43.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"}]: dispatch 2024-03-21T12:53:43.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi043.front.sepia.ceph.com:9093"}]: dispatch 2024-03-21T12:53:43.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:43.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T12:53:43.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-21T12:53:43.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi155.front.sepia.ceph.com:9095"}]: dispatch 2024-03-21T12:53:43.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi155.front.sepia.ceph.com:9095"}]: dispatch 2024-03-21T12:53:43.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:43.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-21T12:53:43.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-21T12:53:43.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi155.front.sepia.ceph.com:3000"}]: dispatch 2024-03-21T12:53:43.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi155.front.sepia.ceph.com:3000"}]: dispatch 2024-03-21T12:53:43.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:43.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:43 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:53:44.556 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:44 smithi155 ceph-mon[33846]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:44.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:44 smithi043 ceph-mon[30062]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:44.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:44 smithi043 ceph-mon[41699]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:46.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:46 smithi155 ceph-mon[33846]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:46.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:46 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:46.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:46 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:46.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:46 smithi043 ceph-mon[30062]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:46.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:46 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:46.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:46 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:46.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:46 smithi043 ceph-mon[41699]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:46.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:46 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:46.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:46 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:47.233 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 12:53:46 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a[80851]: ts=2024-03-21T12:53:46.975Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003561811s 2024-03-21T12:53:48.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:53:48 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:53:48.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:48 smithi155 ceph-mon[33846]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:48.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:48 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:48.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:48 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:48.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:48 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:53:48.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:48 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:53:48.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:48 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:48.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:48 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:53:48.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:53:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:53:48.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:48 smithi043 ceph-mon[30062]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:48.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:48 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:48.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:48 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:48.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:48 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:53:48.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:48 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:53:48.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:48 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:48.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:48 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:53:48.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:48 smithi043 ceph-mon[41699]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:48.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:48 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:48.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:48 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:48.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:48 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:53:48.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:48 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:53:48.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:48 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:53:48.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:48 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:53:49.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:49 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:53:49.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:49 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:53:49.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:49 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:53:50.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:50 smithi155 ceph-mon[33846]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:50.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:50 smithi043 ceph-mon[30062]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:50.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:50 smithi043 ceph-mon[41699]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:52.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:52 smithi155 ceph-mon[33846]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:52.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:52 smithi043 ceph-mon[30062]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:52.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:52 smithi043 ceph-mon[41699]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:54.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:54 smithi155 ceph-mon[33846]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:54.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:54 smithi043 ceph-mon[30062]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:54.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:54 smithi043 ceph-mon[41699]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:53:56.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:56 smithi155 ceph-mon[33846]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:56.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:56 smithi043 ceph-mon[30062]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:56.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:56 smithi043 ceph-mon[41699]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:58.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:53:58 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:53:58.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:53:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:53:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:53:58.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:58 smithi155 ceph-mon[33846]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:58.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:58 smithi043 ceph-mon[30062]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:58.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:58 smithi043 ceph-mon[41699]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:53:59.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:53:59 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:53:59.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:53:59 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:53:59.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:53:59 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:54:00.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:00 smithi155 ceph-mon[33846]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:54:00.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:00 smithi043 ceph-mon[30062]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:54:00.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:00 smithi043 ceph-mon[41699]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:54:02.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:02 smithi155 ceph-mon[33846]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:54:02.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:02 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:54:02.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:02 smithi043 ceph-mon[30062]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:54:02.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:02 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:54:02.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:02 smithi043 ceph-mon[41699]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:54:02.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:02 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:54:04.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:04 smithi155 ceph-mon[33846]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:54:04.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:04 smithi043 ceph-mon[30062]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:54:04.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:04 smithi043 ceph-mon[41699]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:54:06.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:06 smithi155 ceph-mon[33846]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:54:06.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:06 smithi043 ceph-mon[30062]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:54:06.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:06 smithi043 ceph-mon[41699]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:54:08.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:54:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:54:08.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:54:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:54:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:54:08.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:08 smithi155 ceph-mon[33846]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:54:08.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:08 smithi043 ceph-mon[30062]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:54:08.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:08 smithi043 ceph-mon[41699]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:54:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:09 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:54:09.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:09 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:54:09.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:09 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:54:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:10 smithi155 ceph-mon[33846]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:54:10.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:10 smithi043 ceph-mon[30062]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:54:10.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:10 smithi043 ceph-mon[41699]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:54:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:12 smithi155 ceph-mon[33846]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:54:12.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:12 smithi043 ceph-mon[30062]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:54:12.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:12 smithi043 ceph-mon[41699]: pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:54:14.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:14 smithi155 ceph-mon[33846]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:54:14.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:14 smithi043 ceph-mon[30062]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:54:14.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:14 smithi043 ceph-mon[41699]: pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:54:16.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:16 smithi155 ceph-mon[33846]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:54:16.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:16 smithi043 ceph-mon[30062]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:54:16.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:16 smithi043 ceph-mon[41699]: pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:54:17.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:54:17.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:54:17.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:54:18.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:54:18 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:54:18.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:54:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:54:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:54:18.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:18 smithi155 ceph-mon[33846]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:54:18.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:18 smithi043 ceph-mon[30062]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:54:18.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:18 smithi043 ceph-mon[41699]: pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:54:19.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:19 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:54:19.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:19 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:54:19.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:19 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:54:20.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:20 smithi155 ceph-mon[33846]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:54:20.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:20 smithi043 ceph-mon[30062]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:54:20.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:20 smithi043 ceph-mon[41699]: pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:54:22.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:22 smithi155 ceph-mon[33846]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:54:22.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:22 smithi043 ceph-mon[30062]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:54:22.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:22 smithi043 ceph-mon[41699]: pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:54:24.164 INFO:tasks.workunit.client.0.smithi043.stderr:Updating files: 95% (11659/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-03-21T12:54:24.451 INFO:tasks.workunit.client.0.smithi043.stderr:Note: switching to '960fbbe4645c2becc8084669be194c95183c7753'. 2024-03-21T12:54:24.451 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-03-21T12:54:24.451 INFO:tasks.workunit.client.0.smithi043.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-21T12:54:24.451 INFO:tasks.workunit.client.0.smithi043.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-21T12:54:24.451 INFO:tasks.workunit.client.0.smithi043.stderr:state without impacting any branches by switching back to a branch. 2024-03-21T12:54:24.451 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-03-21T12:54:24.452 INFO:tasks.workunit.client.0.smithi043.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-21T12:54:24.452 INFO:tasks.workunit.client.0.smithi043.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-21T12:54:24.452 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-03-21T12:54:24.452 INFO:tasks.workunit.client.0.smithi043.stderr: git switch -c 2024-03-21T12:54:24.452 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-03-21T12:54:24.452 INFO:tasks.workunit.client.0.smithi043.stderr:Or undo this operation with: 2024-03-21T12:54:24.452 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-03-21T12:54:24.452 INFO:tasks.workunit.client.0.smithi043.stderr: git switch - 2024-03-21T12:54:24.452 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-03-21T12:54:24.452 INFO:tasks.workunit.client.0.smithi043.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-21T12:54:24.452 INFO:tasks.workunit.client.0.smithi043.stderr: 2024-03-21T12:54:24.452 INFO:tasks.workunit.client.0.smithi043.stderr:HEAD is now at 960fbbe4645 Merge branch 'cv-refactor-osd-objectstore' of https://github.com/guits/ceph into wip-guits-testing-2024-03-21-0818 2024-03-21T12:54:24.456 DEBUG:teuthology.orchestra.run.smithi043:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-21T12:54:24.490 INFO:tasks.workunit.client.0.smithi043.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-21T12:54:24.492 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-21T12:54:24.492 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-21T12:54:24.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:24 smithi155 ceph-mon[33846]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:54:24.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:24 smithi043 ceph-mon[30062]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:54:24.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:24 smithi043 ceph-mon[41699]: pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:54:25.292 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-21T12:54:25.360 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-21T12:54:25.400 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-21T12:54:25.401 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-21T12:54:25.401 INFO:tasks.workunit.client.0.smithi043.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-21T12:54:25.439 INFO:tasks.workunit.client.0.smithi043.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-21T12:54:25.443 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-03-21T12:54:25.443 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-21T12:54:25.498 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-03-21T12:54:25.501 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-03-21T12:54:25.501 DEBUG:teuthology.orchestra.run.smithi043:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=960fbbe4645c2becc8084669be194c95183c7753 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-03-21T12:54:25.569 INFO:tasks.workunit.client.0.smithi043.stderr:+ parallel=1 2024-03-21T12:54:25.569 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' '' = --serial ']' 2024-03-21T12:54:25.569 INFO:tasks.workunit.client.0.smithi043.stderr:+ crimson=0 2024-03-21T12:54:25.569 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' '' = --crimson ']' 2024-03-21T12:54:25.569 INFO:tasks.workunit.client.0.smithi043.stderr:+ color= 2024-03-21T12:54:25.569 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -t 1 ']' 2024-03-21T12:54:25.569 INFO:tasks.workunit.client.0.smithi043.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-03-21T12:54:25.569 INFO:tasks.workunit.client.0.smithi043.stderr:+ GTEST_OUTPUT_DIR=/home/ubuntu/cephtest/archive/unit_test_xml_report 2024-03-21T12:54:25.569 INFO:tasks.workunit.client.0.smithi043.stderr:+ mkdir -p /home/ubuntu/cephtest/archive/unit_test_xml_report 2024-03-21T12:54:25.569 INFO:tasks.workunit.client.0.smithi043.stderr:+ declare -A pids 2024-03-21T12:54:25.569 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.570 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.570 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_aio 2024-03-21T12:54:25.570 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_aio' 2024-03-21T12:54:25.571 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_aio 2024-03-21T12:54:25.571 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.571 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_aio 2024-03-21T12:54:25.571 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81575 2024-03-21T12:54:25.571 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_aio on pid 81575' 2024-03-21T12:54:25.571 INFO:tasks.workunit.client.0.smithi043.stdout:test api_aio on pid 81575 2024-03-21T12:54:25.575 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81575 2024-03-21T12:54:25.575 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.575 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.575 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-03-21T12:54:25.575 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_aio_pp 2024-03-21T12:54:25.575 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_aio_pp' 2024-03-21T12:54:25.575 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_aio_pp 2024-03-21T12:54:25.575 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.575 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.575 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.576 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.576 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.576 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.576 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.576 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.576 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.576 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.576 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.576 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.576 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.576 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.576 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.576 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.576 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.577 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.577 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.577 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.577 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.577 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.577 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.577 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.577 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.577 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.577 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_aio_pp 2024-03-21T12:54:25.577 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81582 2024-03-21T12:54:25.577 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_aio_pp on pid 81582' 2024-03-21T12:54:25.578 INFO:tasks.workunit.client.0.smithi043.stdout:test api_aio_pp on pid 81582 2024-03-21T12:54:25.582 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81582 2024-03-21T12:54:25.582 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.582 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.582 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-03-21T12:54:25.582 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_io 2024-03-21T12:54:25.583 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_io' 2024-03-21T12:54:25.583 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_io 2024-03-21T12:54:25.583 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.583 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.583 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.583 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.583 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.583 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.583 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.583 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.583 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.583 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.583 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.584 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.584 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.584 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.584 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.584 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.584 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.584 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.584 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.584 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.584 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.584 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.584 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.584 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.584 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.584 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.585 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.585 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.585 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.585 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.585 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.585 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.585 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.585 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.585 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.585 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.585 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.585 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.585 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.585 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.586 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.586 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.586 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.586 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.586 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.586 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.586 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_io 2024-03-21T12:54:25.586 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81591 2024-03-21T12:54:25.586 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_io on pid 81591' 2024-03-21T12:54:25.586 INFO:tasks.workunit.client.0.smithi043.stdout:test api_io on pid 81591 2024-03-21T12:54:25.591 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81591 2024-03-21T12:54:25.591 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.591 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-03-21T12:54:25.591 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.591 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_io_pp 2024-03-21T12:54:25.591 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_io_pp' 2024-03-21T12:54:25.591 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_io_pp 2024-03-21T12:54:25.591 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.591 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.591 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.591 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.592 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.592 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.592 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.592 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.592 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.592 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.592 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.592 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.592 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.592 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.592 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.592 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.592 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.592 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.593 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.593 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.593 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.593 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.593 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.593 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.593 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.593 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.593 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.593 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.593 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.593 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.593 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.593 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.594 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.594 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.594 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.594 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.594 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.594 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.594 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.594 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.594 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.594 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.594 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.594 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.595 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.595 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.595 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.595 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.595 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.595 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.595 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.595 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.595 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.595 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.595 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.595 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_io_pp 2024-03-21T12:54:25.595 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81603 2024-03-21T12:54:25.596 INFO:tasks.workunit.client.0.smithi043.stdout:test api_io_pp on pid 81603 2024-03-21T12:54:25.602 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_io_pp on pid 81603' 2024-03-21T12:54:25.602 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81603 2024-03-21T12:54:25.602 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-03-21T12:54:25.602 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.602 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.602 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_asio 2024-03-21T12:54:25.602 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_asio' 2024-03-21T12:54:25.602 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.602 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.602 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.602 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.602 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.603 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.603 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.603 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_asio 2024-03-21T12:54:25.603 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.603 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.603 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.603 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.603 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.603 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.603 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.603 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.603 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.603 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.603 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.603 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.604 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.604 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.604 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.604 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.604 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.604 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.604 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.604 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.604 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.604 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.604 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.604 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.604 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.604 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.604 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.605 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.605 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.605 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.605 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.605 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.605 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.605 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.605 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.605 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.605 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.605 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.605 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.605 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.606 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.606 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.606 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.606 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.606 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.606 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.606 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.606 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.606 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.606 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.606 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.606 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.606 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.606 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.607 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.607 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.607 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.607 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.607 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.607 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.607 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.607 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.607 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.607 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.607 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.607 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.607 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.607 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.607 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.608 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.608 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.608 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.608 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_asio 2024-03-21T12:54:25.608 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81620 2024-03-21T12:54:25.608 INFO:tasks.workunit.client.0.smithi043.stdout:test api_asio on pid 81620 2024-03-21T12:54:25.614 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_asio on pid 81620' 2024-03-21T12:54:25.614 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81620 2024-03-21T12:54:25.614 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.614 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.614 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-03-21T12:54:25.614 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_list 2024-03-21T12:54:25.614 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.614 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.614 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.614 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.615 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.615 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.615 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.615 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.615 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.615 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.615 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.615 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.615 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.615 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.615 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.615 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.615 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.615 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.616 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.616 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.616 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.616 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.616 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.616 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.616 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.616 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.616 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.616 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.616 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.616 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.616 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.617 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.617 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_list' 2024-03-21T12:54:25.617 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.617 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.617 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.617 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.617 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.617 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.617 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.617 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.617 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.617 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.617 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.617 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.618 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.618 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.618 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.618 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.618 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.618 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.618 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.618 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.618 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.618 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.618 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.618 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.618 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.618 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.619 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.619 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.619 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.619 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.619 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.619 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.619 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.619 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.619 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.619 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.619 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_list 2024-03-21T12:54:25.619 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81575/exe 2024-03-21T12:54:25.619 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.619 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.619 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.620 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.620 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.620 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.620 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.620 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.620 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.620 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.620 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.620 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.620 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.620 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.620 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.620 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.620 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.620 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.621 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.621 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.621 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.621 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.621 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.621 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.621 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.621 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.621 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.621 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.621 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.621 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.621 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.621 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.621 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.622 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.622 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.622 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.622 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.622 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.622 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.622 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.622 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.622 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.622 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.622 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_list 2024-03-21T12:54:25.622 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2024-03-21T12:54:25.622 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81639 2024-03-21T12:54:25.623 INFO:tasks.workunit.client.0.smithi043.stdout:test api_list on pid 81639 2024-03-21T12:54:25.625 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.625 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_list on pid 81639' 2024-03-21T12:54:25.625 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81639 2024-03-21T12:54:25.625 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.626 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.626 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.626 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.626 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-03-21T12:54:25.626 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.626 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.626 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.626 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.626 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_aio: /' 2024-03-21T12:54:25.626 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.626 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.626 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.626 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.626 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.627 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.627 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_lock 2024-03-21T12:54:25.627 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.627 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.627 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.627 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.627 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.627 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.627 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.627 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.627 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.627 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.627 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.627 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.627 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.628 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_lock' 2024-03-21T12:54:25.628 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.628 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.628 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.628 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.628 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.628 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.628 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.628 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.628 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.628 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.628 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.628 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_aio.log 2024-03-21T12:54:25.628 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.628 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.629 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.629 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.629 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.629 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.629 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.629 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_lock 2024-03-21T12:54:25.629 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.629 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.629 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.629 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.629 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.629 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.629 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.629 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.629 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.630 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.630 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.630 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.630 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.630 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.630 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.630 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.630 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.630 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.630 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.630 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.630 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.630 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.630 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.631 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.631 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.631 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.631 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.631 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.631 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.631 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.631 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81582/exe 2024-03-21T12:54:25.631 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.631 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.631 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.631 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.631 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.631 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.632 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.632 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.632 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.632 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.632 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.632 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.632 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.632 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.632 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.632 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.632 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.632 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.632 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.632 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.632 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.633 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.633 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.633 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.633 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.633 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.633 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.633 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_lock 2024-03-21T12:54:25.633 INFO:tasks.workunit.client.0.smithi043.stdout:test api_lock on pid 81655 2024-03-21T12:54:25.636 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.636 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.636 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81655 2024-03-21T12:54:25.637 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_lock on pid 81655' 2024-03-21T12:54:25.637 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.637 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81655 2024-03-21T12:54:25.637 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.637 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.637 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.637 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.637 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-03-21T12:54:25.637 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.637 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.637 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.637 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.637 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.637 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.638 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.638 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.638 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.638 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.638 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_lock_pp 2024-03-21T12:54:25.638 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.638 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.638 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.638 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.638 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.638 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.638 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.638 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.638 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.638 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.638 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.639 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.639 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.639 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.639 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.639 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.639 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_lock_pp' 2024-03-21T12:54:25.639 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.639 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.639 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.639 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.639 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.639 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.639 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.639 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.640 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.640 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.640 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.640 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.640 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.640 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.640 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.640 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.640 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.640 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.640 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_lock_pp 2024-03-21T12:54:25.640 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.640 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2024-03-21T12:54:25.640 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.641 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-03-21T12:54:25.641 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81591/exe 2024-03-21T12:54:25.641 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.641 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.641 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.641 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.641 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.641 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.641 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.641 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.641 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.641 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.641 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.641 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.641 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.642 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.642 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.642 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.642 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.642 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.642 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.642 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.642 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.642 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.642 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.642 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.642 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.642 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.642 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.642 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.642 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.643 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.643 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.643 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.643 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.643 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.643 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.643 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.643 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.643 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.643 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.643 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.643 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_aio_pp: /' 2024-03-21T12:54:25.643 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.643 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.643 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.643 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.644 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.644 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.644 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.644 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.644 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.644 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.644 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.644 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.644 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_io.log 2024-03-21T12:54:25.644 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.644 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.644 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.644 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.644 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.644 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.645 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.645 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.645 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.645 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.645 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.645 INFO:tasks.workunit.client.0.smithi043.stdout:test api_lock_pp on pid 81678 2024-03-21T12:54:25.649 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.649 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.649 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.649 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.649 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_io: /' 2024-03-21T12:54:25.649 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.649 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.649 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.649 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.649 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.649 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.649 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.649 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.649 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.650 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.650 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_lock_pp 2024-03-21T12:54:25.650 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.650 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.650 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.650 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.650 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.650 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.650 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.650 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.650 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.650 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.650 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.651 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.651 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.651 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.651 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81678 2024-03-21T12:54:25.651 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_lock_pp on pid 81678' 2024-03-21T12:54:25.651 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81678 2024-03-21T12:54:25.651 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.651 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.651 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-03-21T12:54:25.651 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.651 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.651 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.651 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.651 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.651 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.651 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.652 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.652 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.652 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.652 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.652 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.652 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.652 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.652 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.652 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.652 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.652 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.652 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.652 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.652 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.652 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.653 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.653 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.653 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.653 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.653 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.653 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_misc 2024-03-21T12:54:25.653 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.653 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_misc' 2024-03-21T12:54:25.653 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.653 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81603/exe 2024-03-21T12:54:25.653 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.653 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.653 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.653 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.653 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.654 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.654 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.654 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.654 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.654 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.654 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.654 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.654 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.654 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.654 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.654 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.654 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.654 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.654 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.654 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.655 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.655 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.655 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.655 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.655 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.655 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.655 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.655 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.655 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.655 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.655 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.655 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.655 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.655 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.655 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.656 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.656 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.656 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.656 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.656 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.656 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.656 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.656 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.656 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.656 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.656 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.656 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.656 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.656 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.656 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.656 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.657 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2024-03-21T12:54:25.657 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.657 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.657 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.657 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.657 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.657 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.657 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.657 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.657 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.657 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.657 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_misc 2024-03-21T12:54:25.657 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.657 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.657 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.657 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.658 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.658 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.658 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.658 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.658 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.658 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.658 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.658 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-03-21T12:54:25.658 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.658 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.658 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.658 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.658 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.658 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.658 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.659 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.659 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.659 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.659 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.659 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.659 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.659 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.659 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.659 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.659 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.659 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.659 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.659 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.659 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.659 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.659 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.660 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.660 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.660 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.660 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_misc 2024-03-21T12:54:25.660 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.660 INFO:tasks.workunit.client.0.smithi043.stdout:test api_misc on pid 81700 2024-03-21T12:54:25.662 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81700 2024-03-21T12:54:25.662 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_misc on pid 81700' 2024-03-21T12:54:25.662 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81700 2024-03-21T12:54:25.662 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.662 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.662 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.662 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-03-21T12:54:25.662 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_misc_pp 2024-03-21T12:54:25.662 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_io_pp: /' 2024-03-21T12:54:25.663 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_misc_pp' 2024-03-21T12:54:25.663 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_misc_pp 2024-03-21T12:54:25.663 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.663 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.663 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.663 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.663 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.663 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.663 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.663 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.663 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.663 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.663 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.663 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.663 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.664 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.664 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.664 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.664 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.664 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81620/exe 2024-03-21T12:54:25.664 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.664 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.664 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.664 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.664 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.664 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.664 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.664 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.664 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.665 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.665 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.665 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.665 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.665 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.665 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.665 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.665 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.665 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.665 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.665 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.665 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.665 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.665 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.666 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.666 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.666 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.666 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.666 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.666 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.666 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.666 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.666 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.666 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.666 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_misc_pp 2024-03-21T12:54:25.666 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.667 INFO:tasks.workunit.client.0.smithi043.stdout:test api_misc_pp on pid 81714 2024-03-21T12:54:25.671 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.671 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81714 2024-03-21T12:54:25.671 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.671 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_misc_pp on pid 81714' 2024-03-21T12:54:25.671 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.671 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81714 2024-03-21T12:54:25.671 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.671 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.671 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2024-03-21T12:54:25.672 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-03-21T12:54:25.672 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.672 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.672 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.672 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.672 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.672 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.672 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.672 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.672 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.672 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.672 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.672 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.672 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_asio: /' 2024-03-21T12:54:25.673 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.673 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.673 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.673 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.673 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.673 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.673 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.673 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.673 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.673 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.673 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.673 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81639/exe 2024-03-21T12:54:25.674 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.674 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.674 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.674 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.674 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.674 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.674 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.674 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.674 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.674 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.674 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.674 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2024-03-21T12:54:25.674 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.675 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.675 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.675 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.675 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.675 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.675 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.675 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.675 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.675 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.675 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_asio.log 2024-03-21T12:54:25.675 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.675 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.675 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.676 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.676 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.676 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.676 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.676 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.676 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.676 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.676 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.676 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.676 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.676 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.676 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.676 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.677 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.677 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.677 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.677 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.677 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.677 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.677 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.677 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.677 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.677 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.677 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.677 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.677 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.677 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.678 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_list: /' 2024-03-21T12:54:25.678 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.678 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.678 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2024-03-21T12:54:25.678 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.678 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.678 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.678 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.678 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.678 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.678 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.678 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.678 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.678 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.679 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.679 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.679 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.679 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.679 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.679 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.679 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.679 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.679 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.679 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.679 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.679 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.679 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.679 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.679 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.680 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.680 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.680 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.680 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.680 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.680 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.680 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.680 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.680 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.680 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.680 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.680 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.680 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.680 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.681 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.681 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.681 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.681 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.681 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.681 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.681 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.681 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.681 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.681 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.681 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.681 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.681 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.681 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.681 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.681 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.682 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.682 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.682 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.682 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.682 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.682 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.682 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.682 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.682 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.682 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.682 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.682 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.682 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.682 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.683 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.683 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.683 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.683 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.683 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.683 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.683 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.683 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.683 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.683 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.683 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.683 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.683 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.683 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.683 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.684 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_tier_pp 2024-03-21T12:54:25.684 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.684 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.684 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.684 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.684 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.684 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.684 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.684 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.684 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.684 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.684 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.684 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.684 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.685 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.685 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.685 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.685 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.685 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.685 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.685 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.685 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.685 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.685 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.685 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.685 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.685 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_list.log 2024-03-21T12:54:25.685 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_tier_pp' 2024-03-21T12:54:25.685 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.686 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.686 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.686 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.686 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.686 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.686 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.686 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.686 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.686 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.686 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.686 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.686 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.686 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.687 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.687 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.687 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.687 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.687 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.687 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.687 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.687 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.687 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.687 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.687 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.687 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.687 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.687 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.688 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.688 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.688 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.688 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.688 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.688 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.688 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.688 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.688 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.688 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.688 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.688 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.688 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.688 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.689 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.689 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.689 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.689 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.689 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.689 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.689 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.689 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.689 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.689 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.689 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.689 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.689 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.689 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.689 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.690 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.690 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.690 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.690 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.690 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.690 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81678/exe 2024-03-21T12:54:25.690 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.690 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_tier_pp 2024-03-21T12:54:25.691 INFO:tasks.workunit.client.0.smithi043.stdout:test api_tier_pp on pid 81753 2024-03-21T12:54:25.695 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.695 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_tier_pp 2024-03-21T12:54:25.695 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.695 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.695 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.695 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.695 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.695 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81753 2024-03-21T12:54:25.695 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.695 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_tier_pp on pid 81753' 2024-03-21T12:54:25.695 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81753 2024-03-21T12:54:25.696 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.696 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.696 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.696 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.696 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.696 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-03-21T12:54:25.696 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2024-03-21T12:54:25.696 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.696 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.696 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.696 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.696 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.696 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.696 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.696 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.696 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-03-21T12:54:25.697 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.697 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.697 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.697 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.697 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.697 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.697 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.697 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.697 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.697 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.697 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.697 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.697 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.697 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.697 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.697 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.698 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.698 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.698 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.698 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.698 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.698 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.698 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.698 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.698 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.698 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.699 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.699 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.699 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.699 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.699 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.699 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.699 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.699 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.699 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.699 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.699 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.699 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.699 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.700 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81655/exe 2024-03-21T12:54:25.700 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_lock_pp: /' 2024-03-21T12:54:25.700 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.700 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.700 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.700 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.700 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.700 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.700 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.700 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.700 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.700 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.700 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.700 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.700 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.701 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.701 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.701 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.701 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.701 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.701 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2024-03-21T12:54:25.701 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.701 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.701 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.701 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.701 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.701 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.701 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.701 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.702 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.702 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.702 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.702 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.702 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.702 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_pool 2024-03-21T12:54:25.702 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_pool' 2024-03-21T12:54:25.702 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81714/exe 2024-03-21T12:54:25.702 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.702 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.702 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.702 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.702 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.702 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.703 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.703 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.703 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.703 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.703 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.703 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.703 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.703 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.703 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.703 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.703 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_pool 2024-03-21T12:54:25.703 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.703 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.703 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.704 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.704 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_misc_pp: /' 2024-03-21T12:54:25.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.704 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2024-03-21T12:54:25.704 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-03-21T12:54:25.704 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.704 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.704 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_lock: /' 2024-03-21T12:54:25.704 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.704 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.704 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.705 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.705 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.705 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.705 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.705 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.705 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.705 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.705 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.705 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.705 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.705 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.705 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.705 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.705 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.706 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.706 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_lock.log 2024-03-21T12:54:25.706 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.706 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.706 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.706 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.706 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.706 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.706 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.706 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.706 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.707 INFO:tasks.workunit.client.0.smithi043.stdout:test api_pool on pid 81828 2024-03-21T12:54:25.710 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_pool 2024-03-21T12:54:25.710 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81828 2024-03-21T12:54:25.710 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_pool on pid 81828' 2024-03-21T12:54:25.710 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81828 2024-03-21T12:54:25.710 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.710 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.710 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.710 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-03-21T12:54:25.710 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_snapshots 2024-03-21T12:54:25.710 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_snapshots' 2024-03-21T12:54:25.710 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.710 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.711 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.711 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.711 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.711 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.711 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.711 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.711 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.711 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.711 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.711 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.711 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.711 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.712 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.712 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.712 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.712 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.712 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.712 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.712 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.712 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.712 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.712 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.712 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.712 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.712 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.712 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.712 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.713 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.713 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.713 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.713 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.713 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.713 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.713 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.713 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.713 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.713 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.713 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.713 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.713 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.714 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.714 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.714 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.715 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.715 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.715 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.715 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.715 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.715 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.715 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.715 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.715 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.715 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.715 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.715 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.715 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.715 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.715 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.716 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.716 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.716 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.716 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.716 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.716 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.716 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.716 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.716 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.716 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.716 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.716 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.716 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.716 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.716 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.716 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.717 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.717 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.717 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.717 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.717 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81700/exe 2024-03-21T12:54:25.717 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.717 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.717 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.717 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.717 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.717 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.717 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.717 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.717 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.718 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.718 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.718 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.718 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.718 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.718 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.718 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.718 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.718 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.718 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.718 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.718 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.718 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.718 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81753/exe 2024-03-21T12:54:25.719 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.719 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.719 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.719 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.719 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.719 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.719 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.719 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.719 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.719 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.719 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.719 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.719 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.719 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.720 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_snapshots 2024-03-21T12:54:25.720 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.720 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.720 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.720 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.720 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.720 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.720 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.720 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_snapshots 2024-03-21T12:54:25.720 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2024-03-21T12:54:25.721 INFO:tasks.workunit.client.0.smithi043.stdout:test api_snapshots on pid 81883 2024-03-21T12:54:25.723 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81883 2024-03-21T12:54:25.723 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_snapshots on pid 81883' 2024-03-21T12:54:25.723 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81883 2024-03-21T12:54:25.723 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2024-03-21T12:54:25.723 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.723 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.723 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-03-21T12:54:25.723 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_misc: /' 2024-03-21T12:54:25.723 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-03-21T12:54:25.723 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_snapshots_pp 2024-03-21T12:54:25.723 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_snapshots_pp' 2024-03-21T12:54:25.724 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.724 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.724 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.724 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.724 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.724 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.724 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_snapshots_pp 2024-03-21T12:54:25.724 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.724 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_misc.log 2024-03-21T12:54:25.724 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.724 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.724 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.724 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.724 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.724 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.725 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.725 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.725 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.725 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.725 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.725 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.725 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.725 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.725 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.725 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.725 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.725 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.725 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.725 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_tier_pp: /' 2024-03-21T12:54:25.725 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_snapshots_pp 2024-03-21T12:54:25.726 INFO:tasks.workunit.client.0.smithi043.stdout:test api_snapshots_pp on pid 81899 2024-03-21T12:54:25.728 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81899 2024-03-21T12:54:25.728 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_snapshots_pp on pid 81899' 2024-03-21T12:54:25.728 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81899 2024-03-21T12:54:25.728 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.728 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.728 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.728 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.728 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_stat 2024-03-21T12:54:25.728 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.728 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.729 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.729 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.729 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.729 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.729 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.729 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.729 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.729 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.729 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.729 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.729 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_stat' 2024-03-21T12:54:25.729 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.729 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.729 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.730 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.730 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.730 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.730 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.730 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.730 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.730 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.730 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.730 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.730 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.730 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.730 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.730 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-03-21T12:54:25.730 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.731 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.731 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_stat 2024-03-21T12:54:25.731 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.731 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_stat 2024-03-21T12:54:25.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.731 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.732 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.732 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81914 2024-03-21T12:54:25.732 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_stat on pid 81914' 2024-03-21T12:54:25.732 INFO:tasks.workunit.client.0.smithi043.stdout:test api_stat on pid 81914 2024-03-21T12:54:25.737 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81914 2024-03-21T12:54:25.737 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.737 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.737 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.737 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.737 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.737 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.738 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.738 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.738 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.738 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.738 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.738 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.738 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.738 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.738 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.738 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.738 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.738 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-03-21T12:54:25.738 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.739 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.739 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.739 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.739 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.739 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.739 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_stat_pp 2024-03-21T12:54:25.739 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_stat_pp' 2024-03-21T12:54:25.739 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.739 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.739 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.739 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.739 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.739 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.739 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.739 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.740 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.740 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.740 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.740 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.740 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.740 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.740 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.740 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.740 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.740 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.740 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.740 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.741 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.741 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.741 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.741 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.742 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.742 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.742 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.742 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.742 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.742 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.742 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.742 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.742 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.742 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_stat_pp 2024-03-21T12:54:25.742 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.742 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.742 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.743 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.743 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.743 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.743 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.744 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.744 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.744 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.744 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.744 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.744 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.744 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.744 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.744 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.744 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.744 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.744 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.744 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.744 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.744 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.744 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.745 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.745 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.745 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.745 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.745 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.745 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.745 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.745 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.745 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.745 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.745 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.745 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.745 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.745 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.745 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.746 INFO:tasks.workunit.client.0.smithi043.stdout:test api_stat_pp on pid 81946 2024-03-21T12:54:25.754 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_stat_pp 2024-03-21T12:54:25.754 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.754 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81946 2024-03-21T12:54:25.754 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_stat_pp on pid 81946' 2024-03-21T12:54:25.754 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=81946 2024-03-21T12:54:25.754 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.755 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.755 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-03-21T12:54:25.755 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.755 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.755 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.755 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.755 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.755 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.755 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.755 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.755 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.755 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.755 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.755 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.756 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.756 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.756 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.756 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.756 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.756 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.756 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.756 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.756 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.756 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.756 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.756 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.756 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.756 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.756 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.757 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.757 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.757 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.757 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.757 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.757 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.757 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.757 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.757 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.757 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.757 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.757 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.757 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.758 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.758 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.758 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.758 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.758 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.758 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.758 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.758 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.758 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.758 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.758 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.759 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.759 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.759 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.759 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.759 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.759 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.759 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.759 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.759 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.759 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.759 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.759 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.759 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.759 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.759 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.760 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.760 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_watch_notify 2024-03-21T12:54:25.760 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.760 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_watch_notify' 2024-03-21T12:54:25.760 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.760 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.760 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.760 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.760 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.760 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.760 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.760 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.760 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81883/exe 2024-03-21T12:54:25.760 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.760 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.760 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.761 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.761 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.761 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.761 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.761 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.761 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.761 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.761 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.761 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.761 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.761 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.761 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.761 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.761 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.762 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.762 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.762 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.762 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.762 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.762 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.762 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.762 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.762 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.762 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.762 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.762 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.762 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.763 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.763 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.763 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.763 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.763 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.763 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.763 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.763 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.763 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.763 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.763 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.763 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.763 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.763 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.763 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.764 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.764 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.764 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.764 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.764 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.764 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.764 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_watch_notify 2024-03-21T12:54:25.764 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-03-21T12:54:25.764 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2024-03-21T12:54:25.764 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81828/exe 2024-03-21T12:54:25.764 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.764 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.764 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.764 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.764 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.765 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.765 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.765 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.765 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.765 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.765 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.765 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_snapshots: /' 2024-03-21T12:54:25.765 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.765 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.765 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.765 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.765 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.765 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.766 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.766 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.766 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.766 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.766 INFO:tasks.workunit.client.0.smithi043.stdout:test api_watch_notify on pid 82000 2024-03-21T12:54:25.771 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_watch_notify 2024-03-21T12:54:25.771 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82000 2024-03-21T12:54:25.771 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_watch_notify on pid 82000' 2024-03-21T12:54:25.771 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82000 2024-03-21T12:54:25.771 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.771 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.772 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_pool: /' 2024-03-21T12:54:25.772 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-03-21T12:54:25.772 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_pool.log 2024-03-21T12:54:25.772 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2024-03-21T12:54:25.772 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.772 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.772 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.772 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.772 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.772 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.772 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.772 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.772 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.772 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.772 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.773 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.773 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.773 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.773 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.773 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.773 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.773 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.773 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.773 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.773 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.773 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.773 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.773 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.773 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.773 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.774 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.774 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.774 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.774 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.774 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.774 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.774 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.774 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.774 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.774 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.774 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.774 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.775 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.775 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.775 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.775 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.775 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.775 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.775 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.775 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.775 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.775 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.775 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.775 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.775 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.775 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.776 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.776 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.776 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.776 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.776 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.776 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.776 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.776 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.776 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.776 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.776 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.776 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.776 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.776 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.776 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_watch_notify_pp 2024-03-21T12:54:25.777 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.777 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.777 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.777 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_watch_notify_pp' 2024-03-21T12:54:25.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.777 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.777 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.777 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.777 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81899/exe 2024-03-21T12:54:25.778 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.778 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.778 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_watch_notify_pp 2024-03-21T12:54:25.778 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.778 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.778 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.778 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.778 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.778 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.778 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.778 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.778 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.778 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.779 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.779 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.779 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.779 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.779 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.779 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.779 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.779 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.779 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.779 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.779 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.779 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.779 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.779 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.779 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.780 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.780 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.780 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2024-03-21T12:54:25.780 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.780 INFO:tasks.workunit.client.0.smithi043.stdout:test api_watch_notify_pp on pid 82028 2024-03-21T12:54:25.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_watch_notify_pp 2024-03-21T12:54:25.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82028 2024-03-21T12:54:25.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_watch_notify_pp on pid 82028' 2024-03-21T12:54:25.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82028 2024-03-21T12:54:25.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.782 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_cmd 2024-03-21T12:54:25.782 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.782 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.782 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.782 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-03-21T12:54:25.782 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.782 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_cmd' 2024-03-21T12:54:25.783 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.783 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-03-21T12:54:25.783 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.783 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_cmd 2024-03-21T12:54:25.783 INFO:tasks.workunit.client.0.smithi043.stdout:test api_cmd on pid 82042 2024-03-21T12:54:25.788 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_cmd 2024-03-21T12:54:25.788 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82042 2024-03-21T12:54:25.788 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_cmd on pid 82042' 2024-03-21T12:54:25.788 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82042 2024-03-21T12:54:25.788 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.788 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.788 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.788 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_cmd_pp 2024-03-21T12:54:25.789 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.789 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.789 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.789 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.789 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.789 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.789 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.789 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.789 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_cmd_pp' 2024-03-21T12:54:25.789 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.789 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.789 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.789 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.790 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.790 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.790 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.790 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.790 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.790 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.790 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-03-21T12:54:25.790 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.790 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.790 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.790 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.791 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-03-21T12:54:25.791 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.791 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.791 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.792 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.792 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.792 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.792 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.792 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.792 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.793 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.793 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.793 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.793 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.794 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.794 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.794 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.794 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.794 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.794 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.794 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81946/exe 2024-03-21T12:54:25.794 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.794 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.794 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.794 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.794 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.795 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.795 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.795 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.795 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.795 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.795 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.795 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.795 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.795 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.795 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.795 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2024-03-21T12:54:25.795 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.795 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.795 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.796 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.796 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.796 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.796 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.796 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.796 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.796 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.796 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.796 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.796 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.796 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.796 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.796 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.797 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.797 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.797 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.797 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.797 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.797 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_stat_pp: /' 2024-03-21T12:54:25.797 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.797 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.797 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.797 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.797 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.797 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.797 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.797 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.798 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.798 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.798 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.798 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.798 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.798 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.798 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.798 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.798 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.798 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.798 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.798 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.798 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_cmd_pp 2024-03-21T12:54:25.798 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.799 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.799 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.799 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.799 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.799 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.799 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.799 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.799 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.799 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.799 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_cmd_pp 2024-03-21T12:54:25.800 INFO:tasks.workunit.client.0.smithi043.stdout:test api_cmd_pp on pid 82068 2024-03-21T12:54:25.802 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82068 2024-03-21T12:54:25.802 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_cmd_pp on pid 82068' 2024-03-21T12:54:25.802 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82068 2024-03-21T12:54:25.802 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.802 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.802 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_service 2024-03-21T12:54:25.802 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_service' 2024-03-21T12:54:25.802 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.802 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.802 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.802 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.802 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.803 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.803 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.803 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.803 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.803 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.803 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.803 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.803 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.803 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.803 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.803 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.803 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.803 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.804 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.804 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-03-21T12:54:25.804 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-03-21T12:54:25.804 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.804 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.804 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_service 2024-03-21T12:54:25.804 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.804 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.804 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.805 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.805 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.805 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.806 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.806 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.806 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.806 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.806 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.806 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.806 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.806 INFO:tasks.workunit.client.0.smithi043.stdout:test api_service on pid 82087 2024-03-21T12:54:25.808 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.808 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.808 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.809 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_service 2024-03-21T12:54:25.809 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.809 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.809 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.809 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.809 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.809 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.809 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82087 2024-03-21T12:54:25.809 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.809 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.810 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_service on pid 82087' 2024-03-21T12:54:25.810 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.810 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82087 2024-03-21T12:54:25.810 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.810 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.810 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.810 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.810 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.810 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_service_pp 2024-03-21T12:54:25.810 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_service_pp' 2024-03-21T12:54:25.810 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.810 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.811 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.811 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.811 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.811 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_service_pp 2024-03-21T12:54:25.811 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.811 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.811 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-03-21T12:54:25.811 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.811 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.811 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.811 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.811 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.811 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.811 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.811 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.812 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.812 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.812 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.812 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.812 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.812 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.812 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.812 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.812 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.812 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.812 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.812 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.812 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.813 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.813 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.813 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.813 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.813 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.813 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.813 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.813 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.813 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.813 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.813 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.813 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.813 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.813 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.813 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.814 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.814 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.814 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.814 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.814 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.814 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.814 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.814 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.814 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.814 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.814 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.814 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.814 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.815 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.815 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.815 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.815 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.815 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.815 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.815 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.815 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.815 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.815 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.815 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.815 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.815 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.815 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.815 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.816 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.816 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.816 INFO:tasks.workunit.client.0.smithi043.stdout:test api_service_pp on pid 82110 2024-03-21T12:54:25.821 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_service_pp 2024-03-21T12:54:25.821 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82110 2024-03-21T12:54:25.821 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_service_pp on pid 82110' 2024-03-21T12:54:25.821 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82110 2024-03-21T12:54:25.821 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.821 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.821 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/81914/exe 2024-03-21T12:54:25.821 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_c_write_operations 2024-03-21T12:54:25.821 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-03-21T12:54:25.822 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_c_write_operations' 2024-03-21T12:54:25.822 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82000/exe 2024-03-21T12:54:25.822 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.822 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.822 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.822 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.822 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.822 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.822 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.822 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.822 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.822 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.822 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.822 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.823 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.823 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.823 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.823 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.823 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.823 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.823 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.823 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.823 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.823 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.823 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.823 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.823 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.823 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.823 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.823 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.824 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.824 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.824 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.824 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.824 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.824 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.824 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.824 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.824 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.824 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.824 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.824 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.824 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.824 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.825 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.825 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.825 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.825 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.825 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.825 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.825 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.825 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.825 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.825 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_c_write_operations 2024-03-21T12:54:25.825 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.825 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.825 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.825 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.826 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_stat.log 2024-03-21T12:54:25.826 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.826 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.826 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.826 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.826 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.826 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.826 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.826 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.826 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.826 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.826 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.826 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.826 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.826 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.827 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.827 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.827 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.827 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.827 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2024-03-21T12:54:25.827 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.827 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.827 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.827 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_watch_notify: /' 2024-03-21T12:54:25.827 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-03-21T12:54:25.827 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.827 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.828 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.828 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.828 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.828 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.828 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.828 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.828 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.828 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.828 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.828 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.828 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.828 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.828 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.828 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.829 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.829 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_stat: /' 2024-03-21T12:54:25.829 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.829 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.829 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.829 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.829 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.830 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.830 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.830 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.830 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.830 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.830 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.830 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.830 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.830 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.830 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.830 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.830 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.830 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.831 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.831 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.831 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.831 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.831 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.831 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.831 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.831 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.831 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.831 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.831 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.831 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.831 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.831 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.831 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.831 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.832 INFO:tasks.workunit.client.0.smithi043.stdout:test api_c_write_operations on pid 82164 2024-03-21T12:54:25.837 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.837 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.837 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.837 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.837 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.837 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.837 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.838 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_c_write_operations 2024-03-21T12:54:25.838 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82164 2024-03-21T12:54:25.838 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_c_write_operations on pid 82164' 2024-03-21T12:54:25.838 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82164 2024-03-21T12:54:25.838 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.838 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.838 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-03-21T12:54:25.838 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.838 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.838 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.838 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.838 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.838 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2024-03-21T12:54:25.839 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.839 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.839 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.839 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.839 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.839 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.839 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.839 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.839 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.839 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.839 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.840 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.840 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.840 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.840 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.840 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.840 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.840 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.840 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.840 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.840 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.840 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.840 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.840 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.840 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.840 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.840 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.841 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.841 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.841 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.841 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.841 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.841 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.841 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.841 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.841 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.841 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.841 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.841 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.841 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.841 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.841 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.842 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.842 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.842 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.842 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.842 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.842 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.842 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.842 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.842 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.842 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.842 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.842 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.842 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.842 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.842 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.843 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.843 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.843 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.843 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.843 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.843 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.843 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.843 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.843 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_c_read_operations 2024-03-21T12:54:25.843 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.843 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.843 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.843 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.843 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.843 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.844 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.844 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.844 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.844 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.844 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.844 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.844 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.844 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.844 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.844 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.844 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82042/exe 2024-03-21T12:54:25.844 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.844 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.844 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.845 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.845 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.845 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.845 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.845 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.845 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.845 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.845 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.845 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.845 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.845 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_c_read_operations' 2024-03-21T12:54:25.845 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.845 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.845 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.845 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.845 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.846 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.846 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.846 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.846 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.846 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.846 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_c_read_operations 2024-03-21T12:54:25.846 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.846 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.846 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2024-03-21T12:54:25.846 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.846 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.846 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.846 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.847 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.847 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.847 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.847 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.847 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.847 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_c_read_operations 2024-03-21T12:54:25.847 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82204 2024-03-21T12:54:25.847 INFO:tasks.workunit.client.0.smithi043.stdout:test api_c_read_operations on pid 82204 2024-03-21T12:54:25.850 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_c_read_operations on pid 82204' 2024-03-21T12:54:25.850 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82204 2024-03-21T12:54:25.850 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.850 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.850 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s api_cls_remote_reads 2024-03-21T12:54:25.850 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' api_cls_remote_reads' 2024-03-21T12:54:25.850 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-03-21T12:54:25.850 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.850 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.851 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.851 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.851 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.851 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.851 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.851 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.851 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.851 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.851 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.851 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.851 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.851 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.851 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.852 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.852 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.852 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.852 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.852 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.852 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.852 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.852 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.852 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.852 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.852 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.852 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.852 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.852 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.852 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.852 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.853 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.853 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo api_cls_remote_reads 2024-03-21T12:54:25.853 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.853 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.853 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.853 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_cmd.log 2024-03-21T12:54:25.853 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.853 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.853 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.853 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.853 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.853 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.853 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.853 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_cmd: /' 2024-03-21T12:54:25.854 INFO:tasks.workunit.client.0.smithi043.stdout:test api_cls_remote_reads on pid 82227 2024-03-21T12:54:25.857 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.857 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.857 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.857 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.858 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=api_cls_remote_reads 2024-03-21T12:54:25.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.858 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.858 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.858 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.858 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.858 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.858 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82227 2024-03-21T12:54:25.859 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test api_cls_remote_reads on pid 82227' 2024-03-21T12:54:25.859 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82227 2024-03-21T12:54:25.859 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.859 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.859 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.859 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.859 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.859 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.859 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.859 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.859 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cls_remote_reads.xml 2>&1 | tee ceph_test_rados_api_cls_remote_reads.log | sed "s/^/ api_cls_remote_reads: /"' 2024-03-21T12:54:25.859 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.859 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.859 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.860 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.860 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.860 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.860 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.860 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.860 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.861 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.861 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.861 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.862 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.862 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.862 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82087/exe 2024-03-21T12:54:25.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.862 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.862 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s list_parallel 2024-03-21T12:54:25.862 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' list_parallel' 2024-03-21T12:54:25.862 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.863 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.863 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.863 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.863 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.864 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.864 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.864 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.864 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.864 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.864 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.864 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.865 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.865 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.865 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.865 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.865 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.865 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.865 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.865 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.865 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.865 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.865 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.865 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.865 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.865 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.866 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.866 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo list_parallel 2024-03-21T12:54:25.866 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.866 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.866 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.866 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.866 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.866 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.866 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.866 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.866 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.866 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.866 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.866 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.866 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.867 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.867 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.867 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.867 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.867 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.867 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.867 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.867 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.867 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.867 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.867 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.867 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.867 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.867 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.867 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.868 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.868 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.868 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.868 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.868 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.868 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.868 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.868 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.868 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.868 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2024-03-21T12:54:25.868 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.868 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.868 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.868 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.868 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.869 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_service.log 2024-03-21T12:54:25.869 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=list_parallel 2024-03-21T12:54:25.869 INFO:tasks.workunit.client.0.smithi043.stdout:test list_parallel on pid 82268 2024-03-21T12:54:25.880 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82268 2024-03-21T12:54:25.880 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test list_parallel on pid 82268' 2024-03-21T12:54:25.880 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82268 2024-03-21T12:54:25.880 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.880 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.880 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.880 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.880 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.880 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.880 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.880 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.881 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.881 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.881 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.881 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.881 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.881 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.881 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.881 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.881 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.881 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.881 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.881 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.881 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.881 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.881 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-03-21T12:54:25.882 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s open_pools_parallel 2024-03-21T12:54:25.882 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.882 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' open_pools_parallel' 2024-03-21T12:54:25.882 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.882 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_service: /' 2024-03-21T12:54:25.882 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.882 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.882 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.882 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.882 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.882 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo open_pools_parallel 2024-03-21T12:54:25.882 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.883 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.883 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.883 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.883 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.883 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.883 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.883 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.883 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.883 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.883 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.883 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.883 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.883 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.884 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.884 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.884 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.884 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.884 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.884 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.884 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.884 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.884 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.884 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.884 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.884 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.884 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.884 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.884 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.885 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.885 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.885 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.885 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.885 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.885 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.885 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.885 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.885 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.885 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.885 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.885 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.885 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.885 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.885 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.886 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.886 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.886 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.886 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.886 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.886 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.886 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.886 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.886 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.886 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.886 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.886 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.886 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.886 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.887 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.887 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.887 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.887 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.887 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.887 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.887 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.887 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.887 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.887 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.887 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.887 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.887 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.887 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.887 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.888 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.888 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.888 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.888 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.888 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.888 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.888 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.888 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.888 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.888 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.888 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.888 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.888 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.888 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.888 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.888 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.889 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.889 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.889 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.889 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.889 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.889 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.889 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.889 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.889 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.889 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.889 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.889 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.889 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.889 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.890 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.890 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.890 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.890 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.890 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.890 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.890 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.890 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.890 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.890 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.890 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.890 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.890 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.891 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.891 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.891 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.891 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.891 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.891 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.891 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.891 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.891 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.891 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.891 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.891 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.891 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82068/exe 2024-03-21T12:54:25.891 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82028/exe 2024-03-21T12:54:25.892 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.892 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.892 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.892 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.892 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.892 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.892 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.892 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.893 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.893 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.893 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.893 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.893 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.893 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=open_pools_parallel 2024-03-21T12:54:25.893 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82301 2024-03-21T12:54:25.893 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test open_pools_parallel on pid 82301' 2024-03-21T12:54:25.893 INFO:tasks.workunit.client.0.smithi043.stdout:test open_pools_parallel on pid 82301 2024-03-21T12:54:25.896 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82301 2024-03-21T12:54:25.896 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-21T12:54:25.896 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.896 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.896 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.896 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.896 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.897 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.897 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.897 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.897 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s delete_pools_parallel 2024-03-21T12:54:25.897 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.897 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.897 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.897 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.897 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.897 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' delete_pools_parallel' 2024-03-21T12:54:25.897 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-03-21T12:54:25.898 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.898 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.898 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.898 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.898 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82110/exe 2024-03-21T12:54:25.898 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.898 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.898 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.898 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.898 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.898 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.898 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.898 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.898 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.898 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.899 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.899 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.899 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.899 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.899 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.899 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.900 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-03-21T12:54:25.900 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.900 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.900 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.900 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.900 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.901 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.901 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.902 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.902 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.902 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.902 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2024-03-21T12:54:25.902 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-03-21T12:54:25.902 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.902 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.902 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.902 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.902 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.902 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.902 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.902 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo delete_pools_parallel 2024-03-21T12:54:25.902 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.902 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.903 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.903 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-03-21T12:54:25.903 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.903 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.903 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.903 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.903 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.903 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=delete_pools_parallel 2024-03-21T12:54:25.903 INFO:tasks.workunit.client.0.smithi043.stdout:test delete_pools_parallel on pid 82319 2024-03-21T12:54:25.907 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82319 2024-03-21T12:54:25.907 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test delete_pools_parallel on pid 82319' 2024-03-21T12:54:25.907 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82319 2024-03-21T12:54:25.907 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T12:54:25.907 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.907 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2024-03-21T12:54:25.907 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.907 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.907 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.908 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.908 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.908 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.908 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-03-21T12:54:25.908 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s cls 2024-03-21T12:54:25.908 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' cls' 2024-03-21T12:54:25.908 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.908 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.908 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.908 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.908 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.908 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.908 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.909 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.909 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.909 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.909 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.909 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.909 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.909 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.909 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.909 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.909 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.909 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.909 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.909 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.909 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.909 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.910 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.910 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.910 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.910 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.911 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.911 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.911 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.911 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.911 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.911 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.911 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo cls 2024-03-21T12:54:25.911 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-03-21T12:54:25.911 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2024-03-21T12:54:25.911 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-03-21T12:54:25.911 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=cls 2024-03-21T12:54:25.911 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82338 2024-03-21T12:54:25.911 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test cls on pid 82338' 2024-03-21T12:54:25.912 INFO:tasks.workunit.client.0.smithi043.stdout:test cls on pid 82338 2024-03-21T12:54:25.916 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82338 2024-03-21T12:54:25.916 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T12:54:25.916 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.917 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.917 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.917 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.917 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82164/exe 2024-03-21T12:54:25.917 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.917 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.918 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cls 2>&1 | tee ceph_test_neorados_cls.log | sed "s/^/ cls: /"' 2024-03-21T12:54:25.918 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.918 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.918 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.918 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.918 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.918 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.918 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.918 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.918 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.918 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.918 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.918 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.918 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.919 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s cmd 2024-03-21T12:54:25.919 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.919 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.919 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.919 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.919 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.919 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' cmd' 2024-03-21T12:54:25.919 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.919 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.919 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.919 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.919 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.919 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.920 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.920 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.920 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.920 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.920 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.920 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.920 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.920 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.920 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.920 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.920 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.920 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.920 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.921 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.921 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo cmd 2024-03-21T12:54:25.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.921 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.921 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-03-21T12:54:25.921 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.921 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.921 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.922 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_service_pp: /' 2024-03-21T12:54:25.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.922 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.922 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.922 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.922 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.923 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.923 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.923 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.923 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.923 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.923 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.923 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.923 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.923 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.923 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.923 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.923 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.923 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.923 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.923 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.924 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.924 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.924 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.924 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.924 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.924 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.924 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.924 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.924 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.924 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.924 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.924 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.924 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.924 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.925 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.925 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.925 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.925 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.925 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.926 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.926 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82204/exe 2024-03-21T12:54:25.926 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.926 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.926 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.926 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.926 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.926 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.926 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.926 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.926 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.926 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.926 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.926 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.926 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.927 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.927 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.927 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.927 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.927 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.927 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.927 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.927 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.927 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.927 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.927 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.927 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.927 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.927 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.927 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.928 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.928 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.928 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.928 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.928 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.928 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.928 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.928 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.928 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.928 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.928 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.928 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.928 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.928 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.928 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.929 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.929 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.929 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.929 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-03-21T12:54:25.929 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2024-03-21T12:54:25.929 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-03-21T12:54:25.929 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.929 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.929 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.929 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.929 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.929 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.929 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.929 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.930 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.930 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.930 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.930 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.930 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.930 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.930 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.930 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.930 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.930 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.930 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.930 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.930 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.930 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2024-03-21T12:54:25.931 INFO:tasks.workunit.client.0.smithi043.stdout:test cmd on pid 82372 2024-03-21T12:54:25.936 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=cmd 2024-03-21T12:54:25.936 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-03-21T12:54:25.936 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82372 2024-03-21T12:54:25.936 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test cmd on pid 82372' 2024-03-21T12:54:25.936 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82372 2024-03-21T12:54:25.936 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T12:54:25.936 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.937 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.937 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.937 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.937 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.937 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.937 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.937 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.937 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.937 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.937 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cmd 2>&1 | tee ceph_test_neorados_cmd.log | sed "s/^/ cmd: /"' 2024-03-21T12:54:25.937 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.937 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.937 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.938 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.938 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.938 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.938 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.938 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.938 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.938 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.938 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.938 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.938 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.938 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s handler_error 2024-03-21T12:54:25.938 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.938 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.938 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.939 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.939 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.939 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.939 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.939 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.939 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.939 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.939 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.939 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.939 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.939 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.939 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.939 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.939 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.940 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.940 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.940 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.940 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.940 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' handler_error' 2024-03-21T12:54:25.940 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.940 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo handler_error 2024-03-21T12:54:25.940 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.940 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.940 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.940 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.940 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.940 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.941 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.941 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.941 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.941 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.941 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.941 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.941 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.941 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.941 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.941 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.941 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.941 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.941 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.941 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.942 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.942 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.942 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.942 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.942 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.942 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.942 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.942 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.942 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.942 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.942 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.942 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.942 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.943 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.943 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.943 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.943 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.943 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.943 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.943 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.943 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.943 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.943 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.943 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.943 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.943 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.943 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.944 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.944 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.944 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.944 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.944 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.944 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.944 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.944 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.944 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82227/exe 2024-03-21T12:54:25.944 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.944 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.944 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.944 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.944 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.945 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.945 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.945 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.945 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.945 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.945 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.945 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.945 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.945 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.945 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.945 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.945 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.945 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.945 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.945 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.945 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.946 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.946 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.946 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.946 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.946 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.946 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.946 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.946 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.946 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.946 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.946 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.946 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.946 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.947 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.947 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.947 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.947 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.947 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.947 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.947 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.947 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.947 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.947 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.947 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.947 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.947 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.947 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.948 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.948 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.948 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.948 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.948 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.948 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.948 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.948 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.948 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.948 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.948 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.948 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.948 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.948 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.949 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.949 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cls_remote_reads.xml 2024-03-21T12:54:25.949 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.949 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.949 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.949 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.949 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.949 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.949 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.950 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.950 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.950 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.950 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.950 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.950 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.950 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.950 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.950 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.950 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.950 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82268/exe 2024-03-21T12:54:25.950 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.950 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.951 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.951 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.951 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.951 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.951 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.951 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.951 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.951 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.951 INFO:tasks.workunit.client.0.smithi043.stdout:test handler_error on pid 82452 2024-03-21T12:54:25.956 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.956 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.956 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.956 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2024-03-21T12:54:25.956 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.956 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.957 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.957 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.957 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_list_parallel.log 2024-03-21T12:54:25.957 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.957 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.957 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.957 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.957 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.957 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=handler_error 2024-03-21T12:54:25.957 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82452 2024-03-21T12:54:25.957 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test handler_error on pid 82452' 2024-03-21T12:54:25.957 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82452 2024-03-21T12:54:25.957 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T12:54:25.958 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.958 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.958 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.958 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.958 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.958 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.958 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.958 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s io 2024-03-21T12:54:25.958 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' io' 2024-03-21T12:54:25.958 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_api_cls_remote_reads.log 2024-03-21T12:54:25.958 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ api_cls_remote_reads: /' 2024-03-21T12:54:25.958 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.959 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.959 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.959 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.960 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.960 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.960 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.960 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.960 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.960 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.960 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.960 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.960 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.960 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.960 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.960 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.960 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.961 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.961 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.961 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.961 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ list_parallel: /' 2024-03-21T12:54:25.961 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.961 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_handler_error 2>&1 | tee ceph_test_neorados_handler_error.log | sed "s/^/ handler_error: /"' 2024-03-21T12:54:25.961 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.961 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.961 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.961 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.961 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.961 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.961 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.961 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.962 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.962 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.962 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo io 2024-03-21T12:54:25.962 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.962 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.962 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.962 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.962 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.962 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.962 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.962 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.962 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.963 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.963 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.963 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.963 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.963 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.963 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82301/exe 2024-03-21T12:54:25.963 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.963 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.963 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.963 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.963 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.963 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.963 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.963 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.963 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.964 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.964 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.964 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.964 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.964 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.964 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.964 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.964 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.964 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.964 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.964 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.964 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.964 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.964 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.964 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.965 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.965 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.965 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.965 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.965 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.965 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.965 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.965 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.965 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.965 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.965 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.965 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.965 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.966 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.966 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.966 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.966 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.966 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.966 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.966 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.966 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.966 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.966 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.966 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.966 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.966 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.967 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.967 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.967 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.967 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.967 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.967 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.967 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.967 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.967 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.967 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.967 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.967 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.967 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.967 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.967 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.968 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2024-03-21T12:54:25.968 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-03-21T12:54:25.968 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-03-21T12:54:25.968 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.968 INFO:tasks.workunit.client.0.smithi043.stdout:test io on pid 82515 2024-03-21T12:54:25.974 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=io 2024-03-21T12:54:25.974 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82515 2024-03-21T12:54:25.974 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test io on pid 82515' 2024-03-21T12:54:25.974 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82515 2024-03-21T12:54:25.974 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T12:54:25.974 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.974 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_io 2>&1 | tee ceph_test_neorados_io.log | sed "s/^/ io: /"' 2024-03-21T12:54:25.974 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82338/exe 2024-03-21T12:54:25.975 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.975 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.975 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.975 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.975 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.975 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.975 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.975 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.975 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.975 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.975 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.976 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.976 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.976 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.976 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.976 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.976 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.976 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.976 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.976 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.976 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.976 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.976 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.976 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.977 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.977 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.977 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.977 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82319/exe 2024-03-21T12:54:25.977 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.977 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.977 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.977 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.977 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.977 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.977 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.977 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.977 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.977 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.978 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.978 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.978 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.978 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.978 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.979 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.979 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.979 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.979 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.979 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.979 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.979 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.979 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.979 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.979 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.979 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.979 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.979 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.979 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.980 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.980 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.980 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.980 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ cls: /' 2024-03-21T12:54:25.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.980 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_cls 2024-03-21T12:54:25.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.980 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.980 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.980 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.981 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.981 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.981 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.981 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.981 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s ec_io 2024-03-21T12:54:25.981 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.981 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.981 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' ec_io' 2024-03-21T12:54:25.981 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_cls.log 2024-03-21T12:54:25.981 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-03-21T12:54:25.981 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2024-03-21T12:54:25.981 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.982 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.982 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.982 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.982 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.982 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.982 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.982 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.982 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.982 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo ec_io 2024-03-21T12:54:25.982 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:25.982 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.982 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.982 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.983 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-03-21T12:54:25.983 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.983 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.983 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.983 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:25.983 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:25.983 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.983 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:25.983 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.983 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:25.983 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:25.983 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:25.983 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.983 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:25.984 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.984 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:25.984 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:25.984 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:25.984 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.984 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:25.984 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.984 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:25.984 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.984 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=ec_io 2024-03-21T12:54:25.984 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82586 2024-03-21T12:54:25.984 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test ec_io on pid 82586' 2024-03-21T12:54:25.985 INFO:tasks.workunit.client.0.smithi043.stdout:test ec_io on pid 82586 2024-03-21T12:54:25.989 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82586 2024-03-21T12:54:25.989 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T12:54:25.989 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:25.989 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_io 2>&1 | tee ceph_test_neorados_ec_io.log | sed "s/^/ ec_io: /"' 2024-03-21T12:54:25.989 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:25.989 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:25.989 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:25.989 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:25.989 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.989 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:25.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:25.990 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:25.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:25.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:25.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:25.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.990 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:25.990 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:25.991 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.991 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:25.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:25.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:25.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:25.991 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:25.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:25.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:25.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s list 2024-03-21T12:54:25.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:25.991 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:25.992 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:25.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:25.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:25.992 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.992 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:25.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:25.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.992 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:25.992 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.992 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.992 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.993 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.993 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.993 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.993 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.993 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.993 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.993 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.993 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.993 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.993 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.994 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.994 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.994 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.994 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:25.994 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' list' 2024-03-21T12:54:25.994 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.995 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:25.995 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.995 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:25.995 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:25.995 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:25.995 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:25.995 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:25.995 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:25.995 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:25.995 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:25.995 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.995 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:25.996 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:25.996 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:25.996 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:25.996 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:25.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:25.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:25.996 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:25.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:25.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.996 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:25.997 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:25.997 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82372/exe 2024-03-21T12:54:25.997 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.997 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.997 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.997 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.997 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.997 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.997 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.997 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.997 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.997 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.997 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.997 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:25.998 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:25.998 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:25.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:25.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:25.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:25.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:25.998 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:25.998 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:25.998 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_cmd 2024-03-21T12:54:25.998 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo list 2024-03-21T12:54:25.998 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82452/exe 2024-03-21T12:54:25.998 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:25.998 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_cmd.log 2024-03-21T12:54:25.998 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:25.999 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:25.999 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:25.999 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:25.999 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:25.999 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:25.999 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:25.999 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:25.999 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:25.999 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:25.999 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_handler_error.log 2024-03-21T12:54:25.999 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ handler_error: /' 2024-03-21T12:54:25.999 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_handler_error 2024-03-21T12:54:25.999 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:25.999 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:26.000 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:26.000 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.000 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:26.000 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.000 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:26.000 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:26.000 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:26.000 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.000 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:26.000 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.000 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:26.000 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:26.001 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:26.001 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.001 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:26.001 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.001 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:26.001 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.001 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:26.001 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:26.001 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:26.001 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:26.001 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.001 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:26.002 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.002 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:26.002 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:26.002 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.002 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:26.002 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.002 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:26.002 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ cmd: /' 2024-03-21T12:54:26.002 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.002 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:26.002 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.002 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:26.002 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:26.002 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=list 2024-03-21T12:54:26.003 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:26.003 INFO:tasks.workunit.client.0.smithi043.stdout:test list on pid 82663 2024-03-21T12:54:26.010 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82663 2024-03-21T12:54:26.010 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test list on pid 82663' 2024-03-21T12:54:26.010 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82663 2024-03-21T12:54:26.010 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T12:54:26.011 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:26.011 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:26.011 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:26.011 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:26.011 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:26.011 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:26.011 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:26.011 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:26.011 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:26.011 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:26.011 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:26.011 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:26.012 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:26.012 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:26.012 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:26.012 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:26.012 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:26.012 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:26.012 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.012 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:26.012 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.013 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:26.013 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:26.013 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.013 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:26.013 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.013 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:26.013 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:26.013 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82515/exe 2024-03-21T12:54:26.013 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:26.013 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:26.013 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:26.014 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:26.014 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.014 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:26.014 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.014 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:26.014 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:26.014 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.014 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:26.014 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:26.014 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:26.014 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:26.014 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:26.014 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:26.014 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:26.015 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:26.015 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:26.015 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:26.015 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ io: /' 2024-03-21T12:54:26.015 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_io.log 2024-03-21T12:54:26.015 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_list 2>&1 | tee ceph_test_neorados_list.log | sed "s/^/ list: /"' 2024-03-21T12:54:26.015 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s ec_list 2024-03-21T12:54:26.015 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_io 2024-03-21T12:54:26.015 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' ec_list' 2024-03-21T12:54:26.015 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:26.015 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:26.015 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:26.015 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:26.015 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.016 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:26.016 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:26.016 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:26.016 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:26.016 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.016 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:26.016 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.016 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:26.016 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:26.016 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.016 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:26.016 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.016 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:26.016 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.017 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:26.017 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.017 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:26.017 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:26.017 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:26.017 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:26.017 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:26.017 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:26.017 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.017 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:26.017 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.017 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:26.017 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:26.018 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.018 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:26.018 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.018 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:26.018 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:26.018 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:26.018 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.018 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:26.018 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.018 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:26.018 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.018 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:26.018 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:26.019 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:26.019 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:26.019 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:26.019 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:26.019 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:26.019 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:26.019 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:26.019 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:26.019 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:26.019 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:26.019 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:26.019 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:26.019 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:26.019 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:26.020 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:26.020 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.020 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:26.020 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.020 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:26.020 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:26.020 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.020 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:26.020 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.020 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:26.020 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo ec_list 2024-03-21T12:54:26.020 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:26.020 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:26.020 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:26.020 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:26.020 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.021 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:26.021 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.021 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:26.021 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:26.021 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:26.021 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:26.021 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.021 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:26.021 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.021 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:26.021 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:26.021 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82586/exe 2024-03-21T12:54:26.021 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:26.021 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.022 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:26.022 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.022 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:26.022 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.022 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:26.022 INFO:tasks.workunit.client.0.smithi043.stdout:test ec_list on pid 82756 2024-03-21T12:54:26.024 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=ec_list 2024-03-21T12:54:26.024 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:26.024 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82756 2024-03-21T12:54:26.024 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:26.024 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test ec_list on pid 82756' 2024-03-21T12:54:26.024 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82756 2024-03-21T12:54:26.024 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T12:54:26.024 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:26.025 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:26.025 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:26.025 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:26.025 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:26.025 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:26.025 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:26.025 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:26.025 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:26.025 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_ec_io.log 2024-03-21T12:54:26.025 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s misc 2024-03-21T12:54:26.025 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ ec_io: /' 2024-03-21T12:54:26.025 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_ec_io 2024-03-21T12:54:26.025 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' misc' 2024-03-21T12:54:26.025 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_list 2>&1 | tee ceph_test_neorados_ec_list.log | sed "s/^/ ec_list: /"' 2024-03-21T12:54:26.025 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:26.026 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:26.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo misc 2024-03-21T12:54:26.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:26.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:26.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:26.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:26.026 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:26.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:26.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:26.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:26.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.026 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:26.027 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=misc 2024-03-21T12:54:26.027 INFO:tasks.workunit.client.0.smithi043.stdout:test misc on pid 82783 2024-03-21T12:54:26.031 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:26.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.031 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82783 2024-03-21T12:54:26.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:26.031 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test misc on pid 82783' 2024-03-21T12:54:26.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.031 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82783 2024-03-21T12:54:26.031 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:26.031 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T12:54:26.031 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:26.031 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:26.031 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:26.032 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.032 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:26.032 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.032 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:26.032 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.032 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_misc 2>&1 | tee ceph_test_neorados_misc.log | sed "s/^/ misc: /"' 2024-03-21T12:54:26.032 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s pool 2024-03-21T12:54:26.032 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:26.032 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:26.032 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' pool' 2024-03-21T12:54:26.032 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:26.032 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:26.032 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.032 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:26.033 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo pool 2024-03-21T12:54:26.033 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:26.033 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:26.033 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:26.033 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:26.033 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:26.033 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.033 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:26.033 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.033 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:26.033 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:26.033 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.033 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:26.033 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:26.033 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.033 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:26.034 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.034 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:26.034 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:26.034 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:26.034 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:26.034 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.034 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:26.034 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.034 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:26.034 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:26.034 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:26.034 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.034 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:26.034 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.035 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:26.035 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:26.035 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:26.035 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.035 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:26.035 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.035 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:26.035 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.035 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:26.035 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.035 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:26.035 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.035 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:26.035 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:26.035 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:26.036 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.036 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:26.036 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.036 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:26.036 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.036 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=pool 2024-03-21T12:54:26.036 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82815 2024-03-21T12:54:26.036 INFO:tasks.workunit.client.0.smithi043.stdout:test pool on pid 82815 2024-03-21T12:54:26.041 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test pool on pid 82815' 2024-03-21T12:54:26.041 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82815 2024-03-21T12:54:26.041 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T12:54:26.041 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:26.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s read_operations 2024-03-21T12:54:26.042 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' read_operations' 2024-03-21T12:54:26.042 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_pool 2>&1 | tee ceph_test_neorados_pool.log | sed "s/^/ pool: /"' 2024-03-21T12:54:26.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo read_operations 2024-03-21T12:54:26.042 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:26.042 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:26.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:26.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:26.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:26.042 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:26.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:26.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:26.042 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:26.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:26.043 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:26.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:26.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:26.043 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:26.043 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:26.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:26.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.043 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:26.044 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.044 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:26.044 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:26.044 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:26.044 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.044 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:26.044 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.044 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:26.044 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:26.044 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.044 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:26.044 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:26.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:26.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:26.045 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:26.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:26.045 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:26.045 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:26.045 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:26.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:26.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.045 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:26.045 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:26.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.046 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:26.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:26.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:26.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:26.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:26.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:26.046 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:26.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:26.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:26.046 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:26.046 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:26.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:26.047 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:26.047 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:26.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:26.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:26.047 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:26.047 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:26.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:26.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.047 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:26.048 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.048 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:26.048 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:26.048 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:26.048 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.048 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:26.048 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.048 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:26.048 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:26.048 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.048 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:26.048 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.048 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:26.048 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.049 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:26.049 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.049 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:26.049 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:26.049 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:26.049 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:26.049 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:26.049 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.049 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:26.049 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.049 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:26.049 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:26.049 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:26.049 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.050 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:26.050 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.050 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:26.050 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:26.050 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:26.050 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.050 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:26.050 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.050 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:26.050 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.050 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:26.051 INFO:tasks.workunit.client.0.smithi043.stdout:test read_operations on pid 82892 2024-03-21T12:54:26.053 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=read_operations 2024-03-21T12:54:26.053 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82892 2024-03-21T12:54:26.053 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test read_operations on pid 82892' 2024-03-21T12:54:26.053 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82892 2024-03-21T12:54:26.054 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T12:54:26.054 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:26.054 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:26.054 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:26.054 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:26.054 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:26.054 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:26.054 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:26.054 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:26.054 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:26.054 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:26.054 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:26.054 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:26.054 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s snapshots 2024-03-21T12:54:26.054 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:26.055 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:26.055 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:26.055 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:26.055 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:26.055 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:26.055 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' snapshots' 2024-03-21T12:54:26.055 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.055 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:26.055 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.055 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:26.055 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:26.055 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.055 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:26.055 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.055 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:26.056 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:26.056 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:26.056 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:26.056 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:26.056 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:26.056 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:26.056 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:26.056 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:26.056 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:26.056 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:26.056 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:26.056 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:26.056 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.056 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:26.056 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_read_operations 2>&1 | tee ceph_test_neorados_read_operations.log | sed "s/^/ read_operations: /"' 2024-03-21T12:54:26.056 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.057 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:26.057 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:26.057 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.057 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:26.057 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.057 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:26.057 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:26.057 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:26.057 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:26.057 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:26.057 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.057 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:26.057 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.057 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:26.058 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo snapshots 2024-03-21T12:54:26.058 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:26.058 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.058 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:26.058 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:26.058 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:26.058 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:26.058 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:26.058 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.058 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:26.058 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82663/exe 2024-03-21T12:54:26.058 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:26.058 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:26.058 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:26.058 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:26.059 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:26.059 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.059 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:26.059 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.059 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:26.059 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:26.059 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.059 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:26.059 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.059 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:26.059 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:26.059 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:26.059 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:26.059 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.059 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.060 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:26.060 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:26.060 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:26.060 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.060 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:26.060 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.060 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:26.060 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.060 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:26.060 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.060 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:26.060 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:26.060 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:26.060 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:26.061 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82756/exe 2024-03-21T12:54:26.061 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:26.061 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:26.061 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:26.061 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=snapshots 2024-03-21T12:54:26.061 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:26.061 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:26.061 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:26.061 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:26.061 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:26.061 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82911 2024-03-21T12:54:26.061 INFO:tasks.workunit.client.0.smithi043.stdout:test snapshots on pid 82911 2024-03-21T12:54:26.065 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test snapshots on pid 82911' 2024-03-21T12:54:26.065 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:26.065 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82911 2024-03-21T12:54:26.065 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T12:54:26.065 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.065 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:26.065 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:26.065 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.065 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:26.065 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:26.066 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.066 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:26.066 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.066 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:26.066 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_snapshots 2>&1 | tee ceph_test_neorados_snapshots.log | sed "s/^/ snapshots: /"' 2024-03-21T12:54:26.066 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.066 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:26.066 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.066 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:26.066 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:26.066 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_list 2024-03-21T12:54:26.066 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s watch_notify 2024-03-21T12:54:26.067 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' watch_notify' 2024-03-21T12:54:26.067 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_list.log 2024-03-21T12:54:26.067 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:26.067 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:26.067 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ list: /' 2024-03-21T12:54:26.067 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:26.067 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:26.067 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:26.067 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:26.067 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:26.067 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:26.067 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:26.067 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:26.067 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo watch_notify 2024-03-21T12:54:26.067 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:26.068 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:26.068 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:26.068 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:26.068 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.068 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:26.068 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:26.068 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:26.068 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:26.068 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:26.068 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:26.068 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:26.068 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.068 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:26.068 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.068 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:26.069 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:26.069 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:26.069 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.069 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:26.069 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.069 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:26.069 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:26.069 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:26.069 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.069 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:26.069 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.069 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:26.069 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.069 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:26.069 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:26.070 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:26.070 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.070 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:26.070 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.070 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:26.070 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:26.070 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:26.070 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:26.070 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:26.070 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.070 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:26.070 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.070 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:26.071 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.071 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:26.071 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:26.071 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.071 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:26.071 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:26.071 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.071 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:26.071 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.071 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:26.071 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:26.071 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.071 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:26.071 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:26.072 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.072 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:26.072 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.072 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:26.072 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_ec_list 2024-03-21T12:54:26.072 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:26.072 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:26.072 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:26.072 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.072 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:26.072 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.072 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:26.072 INFO:tasks.workunit.client.0.smithi043.stdout:test watch_notify on pid 82936 2024-03-21T12:54:26.075 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=watch_notify 2024-03-21T12:54:26.075 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82936 2024-03-21T12:54:26.075 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test watch_notify on pid 82936' 2024-03-21T12:54:26.075 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82936 2024-03-21T12:54:26.075 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-21T12:54:26.075 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_ec_list.log 2024-03-21T12:54:26.075 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:26.075 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:26.076 INFO:tasks.workunit.client.0.smithi043.stderr:++ printf %25s write_operations 2024-03-21T12:54:26.076 INFO:tasks.workunit.client.0.smithi043.stderr:+ r=' write_operations' 2024-03-21T12:54:26.076 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ ec_list: /' 2024-03-21T12:54:26.076 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:26.076 INFO:tasks.workunit.client.0.smithi043.stderr:++ echo write_operations 2024-03-21T12:54:26.076 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:26.076 INFO:tasks.workunit.client.0.smithi043.stderr:++ awk '{print $1}' 2024-03-21T12:54:26.076 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_watch_notify 2>&1 | tee ceph_test_neorados_watch_notify.log | sed "s/^/ watch_notify: /"' 2024-03-21T12:54:26.076 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:26.076 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:26.076 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:26.076 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:26.076 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:26.076 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:26.076 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:26.076 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:26.077 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:26.077 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:26.077 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:26.077 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:26.077 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.077 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:26.077 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.077 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:26.077 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:26.077 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:26.077 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:26.077 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:26.077 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:26.077 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.078 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.078 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:26.078 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:26.078 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.078 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:26.078 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:26.078 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.078 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:26.078 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.078 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:26.078 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:26.078 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:26.078 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.078 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.079 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:26.079 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:26.079 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:26.079 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.079 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:26.079 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.079 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:26.079 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:26.079 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:26.079 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.079 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:26.079 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.079 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:26.079 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:26.079 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:26.080 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.080 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:26.080 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:26.080 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.080 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:26.080 INFO:tasks.workunit.client.0.smithi043.stderr:+ ff=write_operations 2024-03-21T12:54:26.080 INFO:tasks.workunit.client.0.smithi043.stdout:test write_operations on pid 82949 2024-03-21T12:54:26.093 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82949 2024-03-21T12:54:26.093 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82815/exe 2024-03-21T12:54:26.094 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo 'test write_operations on pid 82949' 2024-03-21T12:54:26.094 INFO:tasks.workunit.client.0.smithi043.stderr:+ pids[$f]=82949 2024-03-21T12:54:26.094 INFO:tasks.workunit.client.0.smithi043.stderr:+ ret=0 2024-03-21T12:54:26.094 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 1 -eq 1 ']' 2024-03-21T12:54:26.094 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T12:54:26.094 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81753 2024-03-21T12:54:26.094 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81753 2024-03-21T12:54:26.094 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:26.094 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:26.094 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:26.094 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:26.094 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:26.094 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:26.095 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:26.095 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:26.095 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:26.095 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.095 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:26.095 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.095 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:26.095 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:26.095 INFO:tasks.workunit.client.0.smithi043.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_write_operations 2>&1 | tee ceph_test_neorados_write_operations.log | sed "s/^/ write_operations: /"' 2024-03-21T12:54:26.095 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.095 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:26.095 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.095 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:26.096 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:26.096 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:26.096 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.096 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:26.096 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.096 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:26.096 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.096 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:26.096 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:26.096 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:26.096 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:26.096 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:26.096 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:26.096 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:26.096 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:26.097 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:26.097 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:26.097 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:26.097 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:26.097 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:26.097 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:26.097 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.097 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:26.097 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.097 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:26.097 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:26.097 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.097 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:26.097 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.097 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:26.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:26.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:26.098 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:26.098 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:26.098 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:26.098 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:26.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:26.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.098 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:26.098 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:26.099 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.099 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ pool: /' 2024-03-21T12:54:26.099 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:26.099 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:26.099 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:26.099 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:26.099 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:26.099 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:26.099 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_pool.log 2024-03-21T12:54:26.099 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:26.099 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:26.099 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:26.099 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_pool 2024-03-21T12:54:26.100 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:26.100 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:26.100 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.100 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:26.100 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.100 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:26.100 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:26.100 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.100 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:26.100 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.100 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:26.100 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:26.100 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -f /etc/bashrc ']' 2024-03-21T12:54:26.100 INFO:tasks.workunit.client.0.smithi043.stderr:+ . /etc/bashrc 2024-03-21T12:54:26.100 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -z '' ']' 2024-03-21T12:54:26.101 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:26.101 INFO:tasks.workunit.client.0.smithi043.stderr:++ BASHRCSOURCED=Y 2024-03-21T12:54:26.101 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:26.101 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.101 INFO:tasks.workunit.client.0.smithi043.stderr:++ shopt -q login_shell 2024-03-21T12:54:26.101 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:26.101 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.101 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:26.101 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.101 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:26.101 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:26.101 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:26.101 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.101 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:26.102 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.102 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:26.102 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:26.102 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:26.102 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.102 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:26.102 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.102 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:26.102 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.102 INFO:tasks.workunit.client.0.smithi043.stderr:+++ umask 2024-03-21T12:54:26.102 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82783/exe 2024-03-21T12:54:26.102 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' 0022 -eq 0 ']' 2024-03-21T12:54:26.102 INFO:tasks.workunit.client.0.smithi043.stderr:++ SHELL=/bin/bash 2024-03-21T12:54:26.103 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.103 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-03-21T12:54:26.103 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.103 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/bash_completion.sh 2024-03-21T12:54:26.103 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-03-21T12:54:26.103 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.103 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-03-21T12:54:26.103 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.103 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/cobbler.sh 2024-03-21T12:54:26.103 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:26.103 INFO:tasks.workunit.client.0.smithi043.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-03-21T12:54:26.103 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.103 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-03-21T12:54:26.103 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.103 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorgrep.sh 2024-03-21T12:54:26.104 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.105 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:26.105 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:26.105 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:26.105 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:26.105 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:26.105 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:26.105 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.106 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:26.106 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.106 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:26.106 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:26.106 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:26.106 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:26.106 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:26.106 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.106 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:26.106 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.106 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:26.106 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.107 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:26.107 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.107 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:26.107 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:26.107 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:26.107 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:26.107 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'grep=grep --color=auto' 2024-03-21T12:54:26.107 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'egrep=egrep --color=auto' 2024-03-21T12:54:26.107 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-03-21T12:54:26.107 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.107 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-03-21T12:54:26.107 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.107 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorls.sh 2024-03-21T12:54:26.107 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' '!' -t 0 ']' 2024-03-21T12:54:26.107 INFO:tasks.workunit.client.0.smithi043.stderr:+++ return 2024-03-21T12:54:26.107 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.107 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-03-21T12:54:26.108 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.108 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-03-21T12:54:26.108 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export S_COLORS=auto 2024-03-21T12:54:26.108 INFO:tasks.workunit.client.0.smithi043.stderr:+++ S_COLORS=auto 2024-03-21T12:54:26.108 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.108 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-03-21T12:54:26.108 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.108 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-03-21T12:54:26.108 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.108 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:26.108 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:26.108 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:26.108 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:26.109 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:26.109 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:26.109 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:26.109 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:26.109 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:26.109 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:26.109 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:26.109 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:26.109 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:26.109 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:26.109 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:26.109 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:26.109 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:26.109 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:26.110 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.110 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:26.110 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.110 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:26.110 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:26.110 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.110 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:26.110 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.110 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:26.110 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:26.110 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:26.110 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:26.110 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:26.111 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:26.111 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:26.111 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:26.111 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:26.111 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82892/exe 2024-03-21T12:54:26.111 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:26.111 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:26.111 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:26.111 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:26.111 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:26.111 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:26.111 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:26.111 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:26.111 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:26.112 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:26.112 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:26.112 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:26.112 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:26.112 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:26.112 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.112 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:26.112 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.112 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:26.112 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:26.112 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.112 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:26.112 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.112 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:26.112 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:26.112 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:26.113 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ misc: /' 2024-03-21T12:54:26.113 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_misc.log 2024-03-21T12:54:26.113 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:26.113 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:26.113 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:26.113 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:26.113 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:26.113 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:26.113 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:26.113 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:26.113 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:26.113 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:26.113 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:26.113 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.114 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:26.114 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:26.114 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:26.114 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.114 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:26.114 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:26.114 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.114 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_read_operations 2024-03-21T12:54:26.114 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82911/exe 2024-03-21T12:54:26.114 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:26.114 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_read_operations.log 2024-03-21T12:54:26.114 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ read_operations: /' 2024-03-21T12:54:26.114 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:26.114 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:26.114 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:26.115 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.115 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:26.115 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.115 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:26.115 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:26.115 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_misc 2024-03-21T12:54:26.115 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.115 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:26.115 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.115 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:26.115 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.115 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:26.115 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.115 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:26.115 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:26.115 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:26.116 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:26.116 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:26.116 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:26.116 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:26.116 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:26.116 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:26.116 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:26.116 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:26.116 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:26.116 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:26.116 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:26.116 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:26.116 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:26.116 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_snapshots.log 2024-03-21T12:54:26.116 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:26.117 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:26.117 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_snapshots 2024-03-21T12:54:26.117 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-03-21T12:54:26.117 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-03-21T12:54:26.117 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-03-21T12:54:26.117 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.117 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-03-21T12:54:26.117 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.117 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-03-21T12:54:26.117 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-03-21T12:54:26.117 INFO:tasks.workunit.client.0.smithi043.stderr:+++ /usr/libexec/grepconf.sh -c 2024-03-21T12:54:26.117 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ snapshots: /' 2024-03-21T12:54:26.117 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:26.117 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:26.117 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:26.118 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:26.118 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:26.118 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:26.118 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:26.118 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:26.118 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:26.118 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:26.118 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:26.118 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:26.118 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.118 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:26.118 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.118 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:26.118 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:26.118 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.118 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:26.119 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.119 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:26.119 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:26.119 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82936/exe 2024-03-21T12:54:26.119 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:26.119 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:26.119 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:26.119 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:26.119 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:26.119 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:26.119 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:26.119 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:26.119 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:26.120 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:26.120 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:26.120 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_watch_notify 2024-03-21T12:54:26.120 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_watch_notify.log 2024-03-21T12:54:26.120 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ watch_notify: /' 2024-03-21T12:54:26.120 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-03-21T12:54:26.120 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-03-21T12:54:26.120 INFO:tasks.workunit.client.0.smithi043.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-03-21T12:54:26.120 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.120 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-03-21T12:54:26.120 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.120 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/debuginfod.sh 2024-03-21T12:54:26.120 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-03-21T12:54:26.120 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.120 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-03-21T12:54:26.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/gawk.sh 2024-03-21T12:54:26.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-03-21T12:54:26.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.121 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/lang.sh 2024-03-21T12:54:26.121 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup 2024-03-21T12:54:26.121 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/locale 2024-03-21T12:54:26.121 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:26.121 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG_backup=en_US.UTF-8 2024-03-21T12:54:26.121 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:26.121 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /etc/locale.conf ']' 2024-03-21T12:54:26.121 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -x /usr/bin/sed ']' 2024-03-21T12:54:26.121 INFO:tasks.workunit.client.0.smithi043.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-03-21T12:54:26.121 INFO:tasks.workunit.client.0.smithi043.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-03-21T12:54:26.121 INFO:tasks.workunit.client.0.smithi043.stderr:++++ export LANG=en_US.UTF-8 2024-03-21T12:54:26.122 INFO:tasks.workunit.client.0.smithi043.stderr:++++ LANG=en_US.UTF-8 2024-03-21T12:54:26.122 INFO:tasks.workunit.client.0.smithi043.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-03-21T12:54:26.122 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-03-21T12:54:26.122 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:26.122 INFO:tasks.workunit.client.0.smithi043.stderr:+++ LANG=en_US.UTF-8 2024-03-21T12:54:26.122 INFO:tasks.workunit.client.0.smithi043.stderr:+++ unset LANG_backup config 2024-03-21T12:54:26.122 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n '' ']' 2024-03-21T12:54:26.122 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-03-21T12:54:26.122 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' dumb = linux ']' 2024-03-21T12:54:26.122 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.122 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-03-21T12:54:26.122 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.122 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/less.sh 2024-03-21T12:54:26.122 INFO:tasks.workunit.client.0.smithi043.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-03-21T12:54:26.123 INFO:tasks.workunit.client.0.smithi043.stderr:++ for i in /etc/profile.d/*.sh 2024-03-21T12:54:26.123 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-03-21T12:54:26.123 INFO:tasks.workunit.client.0.smithi043.stderr:++ '[' '' ']' 2024-03-21T12:54:26.123 INFO:tasks.workunit.client.0.smithi043.stderr:++ . /etc/profile.d/which2.sh 2024-03-21T12:54:26.123 INFO:tasks.workunit.client.0.smithi043.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-03-21T12:54:26.123 INFO:tasks.workunit.client.0.smithi043.stderr:+++++ readlink /proc/82949/exe 2024-03-21T12:54:26.123 INFO:tasks.workunit.client.0.smithi043.stderr:++++ basename /usr/bin/bash 2024-03-21T12:54:26.123 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_declare='declare -f' 2024-03-21T12:54:26.123 INFO:tasks.workunit.client.0.smithi043.stderr:+++ which_opt=-f 2024-03-21T12:54:26.123 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export which_declare 2024-03-21T12:54:26.123 INFO:tasks.workunit.client.0.smithi043.stderr:+++ export -f which 2024-03-21T12:54:26.123 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset i 2024-03-21T12:54:26.123 INFO:tasks.workunit.client.0.smithi043.stderr:++ unset -f pathmunge 2024-03-21T12:54:26.123 INFO:tasks.workunit.client.0.smithi043.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-03-21T12:54:26.123 INFO:tasks.workunit.client.0.smithi043.stderr:+ export PATH 2024-03-21T12:54:26.123 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-03-21T12:54:26.124 INFO:tasks.workunit.client.0.smithi043.stderr:+ unset rc 2024-03-21T12:54:26.124 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph_test_neorados_write_operations 2024-03-21T12:54:26.124 INFO:tasks.workunit.client.0.smithi043.stderr:+ tee ceph_test_neorados_write_operations.log 2024-03-21T12:54:26.124 INFO:tasks.workunit.client.0.smithi043.stderr:+ sed 's/^/ write_operations: /' 2024-03-21T12:54:27.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:26 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4040077567' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-03-21T12:54:27.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:26 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2542191100' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:27.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:26 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1457785255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-82758-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:27.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:26 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1759292229' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi043-82926-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:27.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:26 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4040077567' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-03-21T12:54:27.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:26 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2542191100' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:27.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:26 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1457785255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-82758-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:27.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:26 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1759292229' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi043-82926-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:27.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:26 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4040077567' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-03-21T12:54:27.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:26 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2542191100' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:27.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:26 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1457785255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-82758-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:27.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:26 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1759292229' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi043-82926-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:28.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:54:28 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:54:28.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:54:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:54:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:54:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:31 smithi155 ceph-mon[33846]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:54:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:31 smithi155 ceph-mon[33846]: from='client.34784 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:31 smithi155 ceph-mon[33846]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-82758-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:31 smithi155 ceph-mon[33846]: from='client.24886 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi043-82926-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:32.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:31 smithi043 ceph-mon[30062]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:54:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:31 smithi043 ceph-mon[30062]: from='client.34784 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:31 smithi043 ceph-mon[30062]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-82758-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:31 smithi043 ceph-mon[30062]: from='client.24886 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi043-82926-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:31 smithi043 ceph-mon[41699]: pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T12:54:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:31 smithi043 ceph-mon[41699]: from='client.34784 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:31 smithi043 ceph-mon[41699]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-82758-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:31 smithi043 ceph-mon[41699]: from='client.24886 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi043-82926-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:35.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:34 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2812636881' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-81636-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:35.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:34 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3845243656' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-81674-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:35.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:34 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2737830216' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-81695-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:35.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:34 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2216783032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-81661-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:35.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:34 smithi155 ceph-mon[33846]: pgmap v69: 260 pgs: 128 unknown, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T12:54:35.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:34 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2542191100' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:35.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:34 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:54:35.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:34 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4104285912' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-21T12:54:35.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:34 smithi155 ceph-mon[33846]: osdmap e74: 8 total, 8 up, 8 in 2024-03-21T12:54:35.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:34 smithi155 ceph-mon[33846]: from='client.24685 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-81674-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:35.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:34 smithi155 ceph-mon[33846]: from='client.34679 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-81636-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:35.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:34 smithi155 ceph-mon[33846]: from='client.24679 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-81695-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:35.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:34 smithi155 ceph-mon[33846]: from='client.24664 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-81661-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:35.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:34 smithi155 ceph-mon[33846]: from='client.34784 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2812636881' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-81636-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3845243656' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-81674-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2737830216' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-81695-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2216783032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-81661-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[30062]: pgmap v69: 260 pgs: 128 unknown, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T12:54:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2542191100' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:54:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4104285912' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-21T12:54:35.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[30062]: osdmap e74: 8 total, 8 up, 8 in 2024-03-21T12:54:35.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[30062]: from='client.24685 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-81674-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:35.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[30062]: from='client.34679 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-81636-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:35.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[30062]: from='client.24679 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-81695-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:35.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[30062]: from='client.24664 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-81661-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:35.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[30062]: from='client.34784 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2812636881' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-81636-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3845243656' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-81674-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2737830216' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-81695-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2216783032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-81661-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[41699]: pgmap v69: 260 pgs: 128 unknown, 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T12:54:35.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2542191100' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:35.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:54:35.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4104285912' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-21T12:54:35.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[41699]: osdmap e74: 8 total, 8 up, 8 in 2024-03-21T12:54:35.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[41699]: from='client.24685 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-81674-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:35.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[41699]: from='client.34679 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-81636-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:35.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[41699]: from='client.24679 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-81695-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:35.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[41699]: from='client.24664 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-81661-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:35.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:34 smithi043 ceph-mon[41699]: from='client.34784 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:36.276 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-03-21T12:54:36.276 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [----------] Global test environment set-up. 2024-03-21T12:54:36.276 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [----------] 12 tests from AsioRados 2024-03-21T12:54:36.276 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-03-21T12:54:36.276 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (0 ms) 2024-03-21T12:54:36.276 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-03-21T12:54:36.276 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (1 ms) 2024-03-21T12:54:36.276 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-03-21T12:54:36.276 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (1 ms) 2024-03-21T12:54:36.276 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-03-21T12:54:36.277 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (6 ms) 2024-03-21T12:54:36.277 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-03-21T12:54:36.277 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (7 ms) 2024-03-21T12:54:36.277 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-03-21T12:54:36.277 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (5 ms) 2024-03-21T12:54:36.277 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-03-21T12:54:36.277 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (1 ms) 2024-03-21T12:54:36.277 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-03-21T12:54:36.277 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (1 ms) 2024-03-21T12:54:36.277 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-03-21T12:54:36.277 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (0 ms) 2024-03-21T12:54:36.277 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-03-21T12:54:36.277 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (3 ms) 2024-03-21T12:54:36.277 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-03-21T12:54:36.278 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (1 ms) 2024-03-21T12:54:36.278 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-03-21T12:54:36.278 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (3 ms) 2024-03-21T12:54:36.278 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [----------] 12 tests from AsioRados (29 ms total) 2024-03-21T12:54:36.278 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: 2024-03-21T12:54:36.278 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [----------] Global test environment tear-down 2024-03-21T12:54:36.278 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (10624 ms total) 2024-03-21T12:54:36.278 INFO:tasks.workunit.client.0.smithi043.stdout: api_asio: [ PASSED ] 12 tests. 2024-03-21T12:54:36.624 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-03-21T12:54:36.624 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] Global test environment set-up. 2024-03-21T12:54:36.625 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] 7 tests from LibRadosList 2024-03-21T12:54:36.625 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-03-21T12:54:36.625 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.ListObjects (435 ms) 2024-03-21T12:54:36.625 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-03-21T12:54:36.625 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (32 ms) 2024-03-21T12:54:36.625 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-03-21T12:54:36.625 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset foo1,foo2,foo3 2024-03-21T12:54:36.625 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo1 2024-03-21T12:54:36.625 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo2 2024-03-21T12:54:36.625 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo3 2024-03-21T12:54:36.625 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset foo1,foo4,foo5 2024-03-21T12:54:36.625 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo4 2024-03-21T12:54:36.625 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo5 2024-03-21T12:54:36.625 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo1 2024-03-21T12:54:36.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset foo6,foo7 2024-03-21T12:54:36.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo7 2024-03-21T12:54:36.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo6 2024-03-21T12:54:36.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-03-21T12:54:36.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns1:foo4 2024-03-21T12:54:36.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns1:foo5 2024-03-21T12:54:36.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns2:foo7 2024-03-21T12:54:36.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns2:foo6 2024-03-21T12:54:36.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns1:foo1 2024-03-21T12:54:36.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: :foo1 2024-03-21T12:54:36.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: :foo2 2024-03-21T12:54:36.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: :foo3 2024-03-21T12:54:36.626 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (81 ms) 2024-03-21T12:54:36.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-03-21T12:54:36.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 1 0 2024-03-21T12:54:36.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 10 0 2024-03-21T12:54:36.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 13 0 2024-03-21T12:54:36.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 7 0 2024-03-21T12:54:36.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 14 0 2024-03-21T12:54:36.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 0 0 2024-03-21T12:54:36.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 15 0 2024-03-21T12:54:36.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 11 0 2024-03-21T12:54:36.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 5 0 2024-03-21T12:54:36.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 8 0 2024-03-21T12:54:36.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 6 0 2024-03-21T12:54:36.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 3 0 2024-03-21T12:54:36.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 4 0 2024-03-21T12:54:36.627 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 12 0 2024-03-21T12:54:36.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 9 0 2024-03-21T12:54:36.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 2 0 2024-03-21T12:54:36.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-03-21T12:54:36.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (69 ms) 2024-03-21T12:54:36.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-03-21T12:54:36.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: x cursor=MIN 2024-03-21T12:54:36.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=1 cursor=13:02547ec2:::1:head 2024-03-21T12:54:36.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=10 cursor=13:52ea6a34:::10:head 2024-03-21T12:54:36.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=13 cursor=13:566253c9:::13:head 2024-03-21T12:54:36.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=7 cursor=13:5c6b0b28:::7:head 2024-03-21T12:54:36.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=14 cursor=13:62a1935d:::14:head 2024-03-21T12:54:36.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=0 cursor=13:6cac518f:::0:head 2024-03-21T12:54:36.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=15 cursor=13:863748b0:::15:head 2024-03-21T12:54:36.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=11 cursor=13:89d3ae78:::11:head 2024-03-21T12:54:36.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=5 cursor=13:b29083e3:::5:head 2024-03-21T12:54:36.628 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=8 cursor=13:bd63b0f1:::8:head 2024-03-21T12:54:36.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=6 cursor=13:c4fdafeb:::6:head 2024-03-21T12:54:36.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=3 cursor=13:cfc208b3:::3:head 2024-03-21T12:54:36.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=4 cursor=13:d83876eb:::4:head 2024-03-21T12:54:36.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=12 cursor=13:de5d7c5f:::12:head 2024-03-21T12:54:36.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=9 cursor=13:e960b815:::9:head 2024-03-21T12:54:36.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > oid=2 cursor=13:f905c69b:::2:head 2024-03-21T12:54:36.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: FIRST> seek to MIN oid=1 2024-03-21T12:54:36.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=1 cursor=13:02547ec2:::1:head 2024-03-21T12:54:36.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:02547ec2:::1:head 2024-03-21T12:54:36.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:02547ec2:::1:head -> 1 2024-03-21T12:54:36.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=10 cursor=13:52ea6a34:::10:head 2024-03-21T12:54:36.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:52ea6a34:::10:head 2024-03-21T12:54:36.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:52ea6a34:::10:head -> 10 2024-03-21T12:54:36.629 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=13 cursor=13:566253c9:::13:head 2024-03-21T12:54:36.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:566253c9:::13:head 2024-03-21T12:54:36.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:566253c9:::13:head -> 13 2024-03-21T12:54:36.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=7 cursor=13:5c6b0b28:::7:head 2024-03-21T12:54:36.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:5c6b0b28:::7:head 2024-03-21T12:54:36.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:5c6b0b28:::7:head -> 7 2024-03-21T12:54:36.714 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=14 cursor=13:62a1935d:::14:head 2024-03-21T12:54:36.714 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:62a1935d:::14:head 2024-03-21T12:54:36.714 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:62a1935d:::14:head -> 14 2024-03-21T12:54:36.714 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=0 cursor=13:6cac518f:::0:head 2024-03-21T12:54:36.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:6cac518f:::0:head 2024-03-21T12:54:36.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:6cac518f:::0:head -> 0 2024-03-21T12:54:36.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=15 cursor=13:863748b0:::15:head 2024-03-21T12:54:36.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:863748b0:::15:head 2024-03-21T12:54:36.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:863748b0:::15:head -> 15 2024-03-21T12:54:36.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=11 cursor=13:89d3ae78:::11:head 2024-03-21T12:54:36.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:89d3ae78:::11:head 2024-03-21T12:54:36.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:89d3ae78:::11:head -> 11 2024-03-21T12:54:36.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=5 cursor=13:b29083e3:::5:head 2024-03-21T12:54:36.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:b29083e3:::5:head 2024-03-21T12:54:36.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:b29083e3:::5:head -> 5 2024-03-21T12:54:36.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=8 cursor=13:bd63b0f1:::8:head 2024-03-21T12:54:36.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:bd63b0f1:::8:head 2024-03-21T12:54:36.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:bd63b0f1:::8:head -> 8 2024-03-21T12:54:36.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=6 cursor=13:c4fdafeb:::6:head 2024-03-21T12:54:36.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:c4fdafeb:::6:head 2024-03-21T12:54:36.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:c4fdafeb:::6:head -> 6 2024-03-21T12:54:36.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=3 cursor=13:cfc208b3:::3:head 2024-03-21T12:54:36.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:cfc208b3:::3:head 2024-03-21T12:54:36.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:cfc208b3:::3:head -> 3 2024-03-21T12:54:36.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=4 cursor=13:d83876eb:::4:head 2024-03-21T12:54:36.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:d83876eb:::4:head 2024-03-21T12:54:36.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:d83876eb:::4:head -> 4 2024-03-21T12:54:36.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=12 cursor=13:de5d7c5f:::12:head 2024-03-21T12:54:36.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:de5d7c5f:::12:head 2024-03-21T12:54:36.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:de5d7c5f:::12:head -> 12 2024-03-21T12:54:36.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=9 cursor=13:e960b815:::9:head 2024-03-21T12:54:36.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:e960b815:::9:head 2024-03-21T12:54:36.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:e960b815:::9:head -> 9 2024-03-21T12:54:36.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : oid=2 cursor=13:f905c69b:::2:head 2024-03-21T12:54:36.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:f905c69b:::2:head 2024-03-21T12:54:36.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:f905c69b:::2:head -> 2 2024-03-21T12:54:36.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:02547ec2:::1:head 2024-03-21T12:54:36.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=13:02547ec2:::1:head expected=13:02547ec2:::1:head 2024-03-21T12:54:36.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:02547ec2:::1:head -> 1 2024-03-21T12:54:36.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=1 expected=1 2024-03-21T12:54:36.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:89d3ae78:::11:head 2024-03-21T12:54:36.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=13:89d3ae78:::11:head expected=13:89d3ae78:::11:head 2024-03-21T12:54:36.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:89d3ae78:::11:head -> 11 2024-03-21T12:54:36.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=11 expected=11 2024-03-21T12:54:36.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:863748b0:::15:head 2024-03-21T12:54:36.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=13:863748b0:::15:head expected=13:863748b0:::15:head 2024-03-21T12:54:36.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:863748b0:::15:head -> 15 2024-03-21T12:54:36.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=15 expected=15 2024-03-21T12:54:36.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:6cac518f:::0:head 2024-03-21T12:54:36.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=13:6cac518f:::0:head expected=13:6cac518f:::0:head 2024-03-21T12:54:36.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:6cac518f:::0:head -> 0 2024-03-21T12:54:36.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=0 expected=0 2024-03-21T12:54:36.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:62a1935d:::14:head 2024-03-21T12:54:36.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=13:62a1935d:::14:head expected=13:62a1935d:::14:head 2024-03-21T12:54:36.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:62a1935d:::14:head -> 14 2024-03-21T12:54:36.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=14 expected=14 2024-03-21T12:54:36.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:5c6b0b28:::7:head 2024-03-21T12:54:36.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=13:5c6b0b28:::7:head expected=13:5c6b0b28:::7:head 2024-03-21T12:54:36.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:5c6b0b28:::7:head -> 7 2024-03-21T12:54:36.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=7 expected=7 2024-03-21T12:54:36.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:566253c9:::13:head 2024-03-21T12:54:36.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=13:566253c9:::13:head expected=13:566253c9:::13:head 2024-03-21T12:54:36.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:566253c9:::13:head -> 13 2024-03-21T12:54:36.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=13 expected=13 2024-03-21T12:54:36.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:52ea6a34:::10:head 2024-03-21T12:54:36.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=13:52ea6a34:::10:head expected=13:52ea6a34:::10:head 2024-03-21T12:54:36.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:52ea6a34:::10:head -> 10 2024-03-21T12:54:36.839 INFO:tasks.workunit.client.0.smithi043.stdout: a list_parallel: process_1_[82484]: starting. 2024-03-21T12:54:36.839 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_1_[82484]: creating pool ceph_test_rados_list_parallel.smithi043-82443 2024-03-21T12:54:36.839 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_1_[82484]: created object 0... 2024-03-21T12:54:36.839 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_1_[82484]: created object 25... 2024-03-21T12:54:36.839 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_1_[82484]: created object 49... 2024-03-21T12:54:36.839 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_1_[82484]: finishing. 2024-03-21T12:54:36.840 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_1_[82484]: shutting down. 2024-03-21T12:54:36.840 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_2_[82492]: starting. 2024-03-21T12:54:36.840 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_2_[82492]: listing objects. 2024-03-21T12:54:36.840 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_2_[82492]: listed object 0... 2024-03-21T12:54:36.840 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_2_[82492]: listed object 25... 2024-03-21T12:54:36.840 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_2_[82492]: saw 50 objects 2024-03-21T12:54:36.840 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_2_[82492]: shutting down. 2024-03-21T12:54:36.840 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:36.840 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[83642]: starting. 2024-03-21T12:54:36.840 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[83642]: creating pool ceph_test_rados_list_parallel.smithi043-82443 2024-03-21T12:54:36.840 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[83642]: created object 0... 2024-03-21T12:54:36.840 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[83642]: created object 25... 2024-03-21T12:54:36.840 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[83642]: created object 49... 2024-03-21T12:54:36.840 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[83642]: finishing. 2024-03-21T12:54:36.841 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_3_[83642]: shutting down. 2024-03-21T12:54:36.841 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:36.841 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_4_[83643]: starting. 2024-03-21T12:54:36.841 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_4_[83643]: listing objects. 2024-03-21T12:54:36.841 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_4_[83643]: listed object 0... 2024-03-21T12:54:36.841 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_4_[83643]: listed object 25... 2024-03-21T12:54:36.841 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_4_[83643]: saw 43 objects 2024-03-21T12:54:36.841 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_4_[83643]: shutting down. 2024-03-21T12:54:36.841 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:36.841 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_5_[83644]: starting. 2024-03-21T12:54:36.841 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_5_[83644]: removed 25 objects... 2024-03-21T12:54:36.841 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_5_[83644]: removed half of the objects 2024-03-21T12:54:36.841 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_5_[83644]: removed 50 objects... 2024-03-21T12:54:36.841 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_5_[83644]: removed 50 objects 2024-03-21T12:54:36.841 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_5_[83644]: shutting down. 2024-03-21T12:54:36.842 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:36.842 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:36.842 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[83764]: starting. 2024-03-21T12:54:36.842 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[83764]: creating pool ceph_test_rados_list_parallel.smithi043-82443 2024-03-21T12:54:36.842 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[83764]: created object 0... 2024-03-21T12:54:36.842 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[83764]: created object 25... 2024-03-21T12:54:36.842 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[83764]: created object 49... 2024-03-21T12:54:36.842 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[83764]: finishing. 2024-03-21T12:54:36.842 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_6_[83764]: shutting down. 2024-03-21T12:54:36.842 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:36.842 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:36.842 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[83765]: starting. 2024-03-21T12:54:36.842 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[83765]: listing objects. 2024-03-21T12:54:36.842 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[83765]: listed object 0... 2024-03-21T12:54:36.843 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[83765]: listed object 25... 2024-03-21T12:54:36.843 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[83765]: listed object 50... 2024-03-21T12:54:36.843 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[83765]: saw 56 objects 2024-03-21T12:54:36.843 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_7_[83765]: shutting down. 2024-03-21T12:54:36.843 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:36.843 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:36.843 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_8_[83766]: starting. 2024-03-21T12:54:36.843 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_8_[83766]: added 25 objects... 2024-03-21T12:54:36.843 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_8_[83766]: added half of the objects 2024-03-21T12:54:36.843 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_8_[83766]: added 50 objects... 2024-03-21T12:54:36.843 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_8_[83766]: added 50 objects 2024-03-21T12:54:36.843 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_8_[83766]: shutting down. 2024-03-21T12:54:36.843 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:36.843 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:36.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: pgmap v70: 260 pgs: 123 creating+peering, 132 active+clean, 5 unknown; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T12:54:36.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: pgmap v71: 260 pgs: 123 creating+peering, 132 active+clean, 5 unknown; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T12:54:36.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-82758-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:36.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3675023028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-82022-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-81725-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2221000524' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-81775-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3447372333' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-82055-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1633192675' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-82365-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1077062766' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2508476481' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-82121-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/447562252' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-82129-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/51552538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3122624245' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-81755-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2503819138' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-81984-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1457785255' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi043-82758-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-82758-1"}]: dispatch 2024-03-21T12:54:36.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/669064076' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-81880-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2606932143' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82427-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2739279199' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82299-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:54:36.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2879374379' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-21T12:54:36.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: pgmap v73: 900 pgs: 693 creating+peering, 132 active+clean, 75 unknown; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:54:36.960 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: osdmap e75: 8 total, 8 up, 8 in 2024-03-21T12:54:36.960 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2542191100' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:36.960 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: Health check failed: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:54:36.960 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.24697 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-81725-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.960 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.24700 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-81775-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.960 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.24739 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-82055-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.960 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.34730 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-82022-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.960 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.24790 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-82365-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.960 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.24886 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi043-82926-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:36.960 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.24685 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-81674-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:36.960 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.34679 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-81636-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:36.960 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.24679 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-81695-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:36.960 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.24664 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-81661-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:36.960 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: osdmap e76: 8 total, 8 up, 8 in 2024-03-21T12:54:36.960 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1759292229' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi043-82926-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi043-82926-1"}]: dispatch 2024-03-21T12:54:36.961 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/22953481' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82187-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.961 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi043-82758-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-82758-1"}]: dispatch 2024-03-21T12:54:36.961 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.24709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.961 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.24727 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.961 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.24745 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-82121-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.961 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.34709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-81880-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.961 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.24757 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-82129-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.961 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.34829 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82427-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.961 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.24787 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.961 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.24703 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-81755-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.962 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.34880 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82299-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.962 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.24730 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-81984-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.962 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.34784 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:36.962 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.24886 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi043-82926-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi043-82926-1"}]: dispatch 2024-03-21T12:54:36.962 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.24820 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82187-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.962 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/563675567' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-21T12:54:36.962 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:36 smithi155 ceph-mon[33846]: pgmap v75: 1284 pgs: 693 creating+peering, 132 active+clean, 459 unknown; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:54:36.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: pgmap v70: 260 pgs: 123 creating+peering, 132 active+clean, 5 unknown; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T12:54:36.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: pgmap v71: 260 pgs: 123 creating+peering, 132 active+clean, 5 unknown; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T12:54:36.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-82758-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:36.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3675023028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-82022-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-81725-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2221000524' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-81775-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3447372333' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-82055-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1633192675' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-82365-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1077062766' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2508476481' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-82121-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/447562252' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-82129-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/51552538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3122624245' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-81755-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2503819138' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-81984-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1457785255' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi043-82758-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-82758-1"}]: dispatch 2024-03-21T12:54:36.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/669064076' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-81880-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2606932143' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82427-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2739279199' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82299-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:54:36.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2879374379' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-21T12:54:36.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: pgmap v73: 900 pgs: 693 creating+peering, 132 active+clean, 75 unknown; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:54:36.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: osdmap e75: 8 total, 8 up, 8 in 2024-03-21T12:54:36.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2542191100' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:36.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: Health check failed: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:54:36.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.24697 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-81725-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.24700 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-81775-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.24739 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-82055-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.34730 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-82022-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.24790 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-82365-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.24886 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi043-82926-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:36.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.24685 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-81674-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:36.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.34679 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-81636-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:36.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.24679 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-81695-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:36.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.24664 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-81661-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:36.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: osdmap e76: 8 total, 8 up, 8 in 2024-03-21T12:54:36.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1759292229' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi043-82926-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi043-82926-1"}]: dispatch 2024-03-21T12:54:36.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/22953481' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82187-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi043-82758-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-82758-1"}]: dispatch 2024-03-21T12:54:36.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.24709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.24727 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.24745 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-82121-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.34709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-81880-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.24757 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-82129-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.34829 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82427-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.24787 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.24703 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-81755-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.988 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.34880 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82299-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.989 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.24730 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-81984-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.989 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.34784 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:36.989 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.24886 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi043-82926-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi043-82926-1"}]: dispatch 2024-03-21T12:54:36.989 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.24820 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82187-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.989 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/563675567' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-21T12:54:36.989 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[30062]: pgmap v75: 1284 pgs: 693 creating+peering, 132 active+clean, 459 unknown; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:54:36.989 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: pgmap v70: 260 pgs: 123 creating+peering, 132 active+clean, 5 unknown; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T12:54:36.989 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: pgmap v71: 260 pgs: 123 creating+peering, 132 active+clean, 5 unknown; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T12:54:36.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi043-82758-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:36.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3675023028' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-82022-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-81725-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2221000524' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-81775-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3447372333' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-82055-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1633192675' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-82365-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1077062766' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2508476481' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-82121-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/447562252' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-82129-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/51552538' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.990 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3122624245' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-81755-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2503819138' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-81984-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1457785255' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi043-82758-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-82758-1"}]: dispatch 2024-03-21T12:54:36.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/669064076' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-81880-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2606932143' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82427-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2739279199' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82299-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:54:36.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2879374379' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-21T12:54:36.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: pgmap v73: 900 pgs: 693 creating+peering, 132 active+clean, 75 unknown; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:54:36.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: osdmap e75: 8 total, 8 up, 8 in 2024-03-21T12:54:36.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2542191100' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:36.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: Health check failed: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:54:36.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.24697 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-81725-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.24700 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-81775-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.991 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.24739 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-82055-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.34730 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-82022-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.24790 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-82365-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.24886 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi043-82926-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:36.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.24685 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi043-81674-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:36.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.34679 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi043-81636-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:36.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.24679 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi043-81695-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:36.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.24664 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi043-81661-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:36.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: osdmap e76: 8 total, 8 up, 8 in 2024-03-21T12:54:36.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1759292229' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi043-82926-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi043-82926-1"}]: dispatch 2024-03-21T12:54:36.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/22953481' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82187-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi043-82758-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-82758-1"}]: dispatch 2024-03-21T12:54:36.992 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.24709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.24727 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.24745 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-82121-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.34709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-81880-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.24757 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-82129-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.34829 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82427-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.24787 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.24703 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-81755-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.34880 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82299-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.24730 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-81984-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.34784 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:36.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.24886 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi043-82926-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi043-82926-1"}]: dispatch 2024-03-21T12:54:36.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.24820 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82187-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:36.993 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/563675567' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-21T12:54:36.994 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:36 smithi043 ceph-mon[41699]: pgmap v75: 1284 pgs: 693 creating+peering, 132 active+clean, 459 unknown; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:54:37.092 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ********************** api_service: [==========] Running 4 tests from 1 test suite. 2024-03-21T12:54:37.092 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [----------] Global test environment set-up. 2024-03-21T12:54:37.092 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [----------] 4 tests from LibRadosService 2024-03-21T12:54:37.092 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-03-21T12:54:37.092 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5051 ms) 2024-03-21T12:54:37.092 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-03-21T12:54:37.092 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ OK ] LibRadosService.RegisterLate (28 ms) 2024-03-21T12:54:37.092 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-03-21T12:54:37.093 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: cluster: 2024-03-21T12:54:37.093 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: id: c06c8d1a-e780-11ee-95cd-87774f69a715 2024-03-21T12:54:37.093 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: health: HEALTH_OK 2024-03-21T12:54:37.093 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-03-21T12:54:37.093 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: services: 2024-03-21T12:54:37.093 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-03-21T12:54:37.093 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-03-21T12:54:37.093 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-03-21T12:54:37.093 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-03-21T12:54:37.093 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-03-21T12:54:37.093 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: data: 2024-03-21T12:54:37.093 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: pools: 6 pools, 132 pgs 2024-03-21T12:54:37.093 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: objects: 198 objects, 582 KiB 2024-03-21T12:54:37.093 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: usage: 231 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:54:37.093 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: pgs: 132 active+clean 2024-03-21T12:54:37.094 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-03-21T12:54:37.094 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: io: 2024-03-21T12:54:37.094 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: client: 853 B/s rd, 0 op/s rd, 0 op/s wr 2024-03-21T12:54:37.094 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-03-21T12:54:37.094 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-03-21T12:54:37.094 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: cluster: 2024-03-21T12:54:37.094 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: id: c06c8d1a-e780-11ee-95cd-87774f69a715 2024-03-21T12:54:37.094 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: health: HEALTH_OK 2024-03-21T12:54:37.094 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-03-21T12:54:37.094 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: services: 2024-03-21T12:54:37.094 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-03-21T12:54:37.094 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-03-21T12:54:37.094 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-03-21T12:54:37.094 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-03-21T12:54:37.095 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-03-21T12:54:37.095 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: data: 2024-03-21T12:54:37.095 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: pools: 6 pools, 132 pgs 2024-03-21T12:54:37.095 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: objects: 198 objects, 582 KiB 2024-03-21T12:54:37.095 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: usage: 231 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:54:37.095 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: pgs: 132 active+clean 2024-03-21T12:54:37.095 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-03-21T12:54:37.095 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: io: 2024-03-21T12:54:37.095 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: client: 853 B/s rd, 0 op/s rd, 0 op/s wr 2024-03-21T12:54:37.095 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-03-21T12:54:37.095 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-03-21T12:54:37.095 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: cluster: 2024-03-21T12:54:37.095 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: id: c06c8d1a-e780-11ee-95cd-87774f69a715 2024-03-21T12:54:37.095 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: health: HEALTH_WARN 2024-03-21T12:54:37.096 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 4 pool(s) do not have an application enabled 2024-03-21T12:54:37.096 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-03-21T12:54:37.096 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: services: 2024-03-21T12:54:37.096 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-03-21T12:54:37.096 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-03-21T12:54:37.096 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-03-21T12:54:37.096 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-03-21T12:54:37.096 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-03-21T12:54:37.096 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-03-21T12:54:37.096 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: data: 2024-03-21T12:54:37.096 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: pools: 10 pools, 260 pgs 2024-03-21T12:54:37.096 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: objects: 198 objects, 582 KiB 2024-03-21T12:54:37.096 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: usage: 231 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:54:37.096 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: pgs: 1.923% pgs unknown 2024-03-21T12:54:37.097 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 47.308% pgs not active 2024-03-21T12:54:37.097 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 132 active+clean 2024-03-21T12:54:37.097 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 123 creating+peering 2024-03-21T12:54:37.097 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 5 unknown 2024-03-21T12:54:37.097 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-03-21T12:54:37.097 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: io: 2024-03-21T12:54:37.097 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: client: 1023 B/s rd, 0 op/s rd, 0 op/s wr 2024-03-21T12:54:37.097 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-03-21T12:54:37.097 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-03-21T12:54:37.182 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: ********* 2024-03-21T12:54:37.182 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[83813]: starting. 2024-03-21T12:54:37.182 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[83813]: creating pool ceph_test_rados_list_parallel.smithi043-82443 2024-03-21T12:54:37.182 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[83813]: created object 0... 2024-03-21T12:54:37.182 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[83813]: created object 25... 2024-03-21T12:54:37.182 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[83813]: created object 49... 2024-03-21T12:54:37.182 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[83813]: finishing. 2024-03-21T12:54:37.183 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_9_[83813]: shutting down. 2024-03-21T12:54:37.183 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:37.183 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:37.183 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:37.183 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[83814]: starting. 2024-03-21T12:54:37.183 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[83814]: listing objects. 2024-03-21T12:54:37.183 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[83814]: listed object 0... 2024-03-21T12:54:37.183 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[83814]: listed object 25... 2024-03-21T12:54:37.183 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[83814]: listed object 50... 2024-03-21T12:54:37.183 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[83814]: listed object 75... 2024-03-21T12:54:37.183 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[83814]: saw 98 objects 2024-03-21T12:54:37.183 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_10_[83814]: shutting down. 2024-03-21T12:54:37.183 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:37.183 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:37.184 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:37.184 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_13_[83817]: starting. 2024-03-21T12:54:37.184 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_13_[83817]: removed 25 objects... 2024-03-21T12:54:37.184 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_13_[83817]: removed half of the objects 2024-03-21T12:54:37.184 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_13_[83817]: removed 50 objects... 2024-03-21T12:54:37.184 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_13_[83817]: removed 50 objects 2024-03-21T12:54:37.184 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_13_[83817]: shutting down. 2024-03-21T12:54:37.184 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:37.184 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:37.184 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:37.184 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_11_[83815]: starting. 2024-03-21T12:54:37.184 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_11_[83815]: added 25 objects... 2024-03-21T12:54:37.184 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_11_[83815]: added half of the objects 2024-03-21T12:54:37.184 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_11_[83815]: added 50 objects... 2024-03-21T12:54:37.185 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_11_[83815]: added 50 objects 2024-03-21T12:54:37.185 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_11_[83815]: shutting down. 2024-03-21T12:54:37.185 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:37.185 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:37.185 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:37.185 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_12_[83816]: starting. 2024-03-21T12:54:37.185 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_12_[83816]: added 25 objects... 2024-03-21T12:54:37.185 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_12_[83816]: added half of the objects 2024-03-21T12:54:37.185 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_12_[83816]: added 50 objects... 2024-03-21T12:54:37.185 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_12_[83816]: added 50 objects 2024-03-21T12:54:37.185 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_12_[83816]: shutting down. 2024-03-21T12:54:37.185 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:37.185 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:37.185 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:37.185 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:37.186 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[83894]: starting. 2024-03-21T12:54:37.186 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[83894]: creating pool ceph_test_rados_list_parallel.smithi043-82443 2024-03-21T12:54:37.186 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[83894]: created object 0... 2024-03-21T12:54:37.186 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[83894]: created object 25... 2024-03-21T12:54:37.186 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[83894]: created object 49... 2024-03-21T12:54:37.186 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[83894]: finishing. 2024-03-21T12:54:37.186 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_14_[83894]: shutting down. 2024-03-21T12:54:37.186 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:37.186 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:37.186 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:37.186 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:37.186 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[83895]: starting. 2024-03-21T12:54:37.186 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[83895]: listing objects. 2024-03-21T12:54:37.186 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[83895]: listed object 0... 2024-03-21T12:54:37.239 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[83895]: listed object 2 handler_error: Running main() from gmock_main.cc 2024-03-21T12:54:37.240 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [==========] Running 1 test from 1 test suite. 2024-03-21T12:54:37.240 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [----------] Global test environment set-up. 2024-03-21T12:54:37.240 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [----------] 1 test from neocls_handler_error 2024-03-21T12:54:37.240 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [ RUN ] neocls_handler_error.test_handler_error 2024-03-21T12:54:37.240 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [ OK ] neocls_handler_error.test_handler_error (11291 ms) 2024-03-21T12:54:37.240 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [----------] 1 test from neocls_handler_error (11291 ms total) 2024-03-21T12:54:37.240 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: 2024-03-21T12:54:37.240 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [----------] Global test environment tear-down 2024-03-21T12:54:37.240 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [==========] 1 test from 1 test suite ran. (11291 ms total) 2024-03-21T12:54:37.240 INFO:tasks.workunit.client.0.smithi043.stdout: handler_error: [ PASSED ] 1 test. 2024-03-21T12:54:37.240 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: Running main() from gmock_main.cc 2024-03-21T12:54:37.240 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-03-21T12:54:37.240 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-03-21T12:54:37.240 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-03-21T12:54:37.241 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-03-21T12:54:37.241 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (84 ms) 2024-03-21T12:54:37.241 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-03-21T12:54:37.241 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (55 ms) 2024-03-21T12:54:37.241 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-03-21T12:54:37.241 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (11280 ms) 2024-03-21T12:54:37.241 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (11419 ms total) 2024-03-21T12:54:37.241 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: 2024-03-21T12:54:37.241 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-03-21T12:54:37.241 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (11419 ms total) 2024-03-21T12:54:37.241 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-03-21T12:54:37.245 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [==========] Running 1 test from 1 test suite. 2024-03-21T12:54:37.245 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [----------] Global test environment set-up. 2024-03-21T12:54:37.245 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads 2024-03-21T12:54:37.245 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [ RUN ] ClsTestRemoteReads.TestGather 2024-03-21T12:54:37.245 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [ OK ] ClsTestRemoteReads.TestGather (11352 ms) 2024-03-21T12:54:37.245 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads (11352 ms total) 2024-03-21T12:54:37.246 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: 2024-03-21T12:54:37.246 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [----------] Global test environment tear-down 2024-03-21T12:54:37.246 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [==========] 1 test from 1 test suite ran. (11352 ms total) 2024-03-21T12:54:37.246 INFO:tasks.workunit.client.0.smithi043.stdout: api_cls_remote_reads: [ PASSED ] 1 test. 2024-03-21T12:54:38.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:54:38.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.24697 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-81725-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.24700 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-81775-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.24739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-82055-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.34730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-82022-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.24790 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-82365-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.24709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.24727 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-82121-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.34709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-81880-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.24757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-82129-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.34829 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82427-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.24787 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.24703 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-81755-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.34880 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82299-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.24730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-81984-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.34784 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:38.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.24820 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82187-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2542191100' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-82308-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:38.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: osdmap e77: 8 total, 8 up, 8 in 2024-03-21T12:54:38.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.34784 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-82308-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:38.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3449806415' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:38.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/247888446' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:38.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:38.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi043-82758-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-82758-1"}]': finished 2024-03-21T12:54:38.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.24886 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi043-82926-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi043-82926-1"}]': finished 2024-03-21T12:54:38.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3291474611' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-21T12:54:38.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: osdmap e78: 8 total, 8 up, 8 in 2024-03-21T12:54:38.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:38 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1322313253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-81636-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:38.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:54:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:54:38.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.24697 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-81725-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.24700 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-81775-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.24739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-82055-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.34730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-82022-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.24790 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-82365-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.24709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.24727 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-82121-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.34709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-81880-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.24757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-82129-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.34829 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82427-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.24787 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.24703 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-81755-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.34880 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82299-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.24730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-81984-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.34784 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:38.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.24820 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82187-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.24697 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi043-81725-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.24700 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi043-81775-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.24739 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi043-82055-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.34730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi043-82022-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.24790 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi043-82365-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.24709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.24727 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.24745 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi043-82121-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.34709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi043-81880-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.24757 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi043-82129-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.34829 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82427-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.24787 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.24703 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi043-81755-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.34880 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82299-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.24730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi043-81984-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.34784 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:38.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.24820 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82187-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:38.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2542191100' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-82308-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:38.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: osdmap e77: 8 total, 8 up, 8 in 2024-03-21T12:54:38.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.34784 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-82308-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:38.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3449806415' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:38.487 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/247888446' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:38.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:38.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi043-82758-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-82758-1"}]': finished 2024-03-21T12:54:38.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.24886 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi043-82926-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi043-82926-1"}]': finished 2024-03-21T12:54:38.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3291474611' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-21T12:54:38.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: osdmap e78: 8 total, 8 up, 8 in 2024-03-21T12:54:38.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1322313253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-81636-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:38.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2542191100' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-82308-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:38.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: osdmap e77: 8 total, 8 up, 8 in 2024-03-21T12:54:38.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.34784 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-82308-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:38.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3449806415' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:38.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/247888446' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:38.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:38.489 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi043-82758-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi043-82758-1"}]': finished 2024-03-21T12:54:38.489 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.24886 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi043-82926-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi043-82926-1"}]': finished 2024-03-21T12:54:38.489 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3291474611' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-21T12:54:38.489 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: osdmap e78: 8 total, 8 up, 8 in 2024-03-21T12:54:38.489 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:38 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1322313253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-81636-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:39.944 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: Running main() from gmock_main.cc 2024-03-21T12:54:39.945 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [==========] Running 17 tests from 1 test suite. 2024-03-21T12:54:39.945 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-03-21T12:54:39.945 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest 2024-03-21T12:54:39.945 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-03-21T12:54:39.945 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-03-21T12:54:39.945 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-03-21T12:54:39.945 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (399 ms) 2024-03-21T12:54:39.945 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-03-21T12:54:39.945 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (8 ms) 2024-03-21T12:54:39.945 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-03-21T12:54:39.945 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (11 ms) 2024-03-21T12:54:39.945 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-03-21T12:54:39.945 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (29 ms) 2024-03-21T12:54:39.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-03-21T12:54:39.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (9 ms) 2024-03-21T12:54:39.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-03-21T12:54:39.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (6 ms) 2024-03-21T12:54:39.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-03-21T12:54:39.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (5 ms) 2024-03-21T12:54:39.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-03-21T12:54:39.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (7 ms) 2024-03-21T12:54:39.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-03-21T12:54:39.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (4 ms) 2024-03-21T12:54:39.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-03-21T12:54:39.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (5 ms) 2024-03-21T12:54:39.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-03-21T12:54:39.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (7 ms) 2024-03-21T12:54:39.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat2 2024-03-21T12:54:39.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat2 (5 ms) 2024-03-21T12:54:39.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-03-21T12:54:39.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (11 ms) 2024-03-21T12:54:39.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-03-21T12:54:39.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (9 ms) 2024-03-21T12:54:39.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-03-21T12:54:39.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (9 ms) 2024-03-21T12:54:39.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-03-21T12:54:39.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (3 ms) 2024-03-21T12:54:39.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest (527 ms total) 2024-03-21T12:54:39.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: 2024-03-21T12:54:39.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-03-21T12:54:39.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [==========] 17 tests from 1 test suite ran. (14101 ms total) 2024-03-21T12:54:39.948 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_read_operations: [ PASSED ] 17 tests. 2024-03-21T12:54:39.948 INFO:tasks.workunit.client.0.smithi043.stdout:5... 2024-03-21T12:54:39.948 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[83895]: listed object 50... 2024-03-21T12:54:39.948 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[83895]: listed object 75... 2024-03-21T12:54:39.948 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[83895]: listed object 100... 2024-03-21T12:54:39.948 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[83895]: listed object 125... 2024-03-21T12:54:39.948 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[83895]: saw 150 objects 2024-03-21T12:54:39.948 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_15_[83895]: shutting down. 2024-03-21T12:54:39.949 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:39.949 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:39.949 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:39.949 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:39.949 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_16_[83896]: starting. 2024-03-21T12:54:39.949 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_16_[83896]: added 25 objects... 2024-03-21T12:54:39.949 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_16_[83896]: added half of the objects 2024-03-21T12:54:39.949 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_16_[83896]: added 50 objects... 2024-03-21T12:54:39.949 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_16_[83896]: added 50 objects 2024-03-21T12:54:39.949 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: process_16_[83896]: shutting down. 2024-03-21T12:54:39.949 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:39.949 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:39.949 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:39.949 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:39.949 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******************************* 2024-03-21T12:54:39.949 INFO:tasks.workunit.client.0.smithi043.stdout: list_parallel: ******* SUCCESS ********** 2024-03-21T12:54:40.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:39 smithi155 ceph-mon[33846]: pgmap v78: 884 pgs: 337 creating+peering, 132 active+clean, 415 unknown; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 971 B/s rd, 0 op/s 2024-03-21T12:54:40.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:39 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:40.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:39 smithi155 ceph-mon[33846]: from='client.25114 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:40.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:39 smithi155 ceph-mon[33846]: from='client.35132 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-81636-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:40.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:39 smithi155 ceph-mon[33846]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:40.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:39 smithi155 ceph-mon[33846]: from='client.34784 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-82308-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]': finished 2024-03-21T12:54:40.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:39 smithi043 ceph-mon[41699]: pgmap v78: 884 pgs: 337 creating+peering, 132 active+clean, 415 unknown; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 971 B/s rd, 0 op/s 2024-03-21T12:54:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:39 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:39 smithi043 ceph-mon[41699]: from='client.25114 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:39 smithi043 ceph-mon[41699]: from='client.35132 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-81636-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:39 smithi043 ceph-mon[41699]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:39 smithi043 ceph-mon[41699]: from='client.34784 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-82308-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]': finished 2024-03-21T12:54:40.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:39 smithi043 ceph-mon[30062]: pgmap v78: 884 pgs: 337 creating+peering, 132 active+clean, 415 unknown; 582 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 971 B/s rd, 0 op/s 2024-03-21T12:54:40.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:39 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:40.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:39 smithi043 ceph-mon[30062]: from='client.25114 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:40.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:39 smithi043 ceph-mon[30062]: from='client.35132 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-81636-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:40.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:39 smithi043 ceph-mon[30062]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:40.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:39 smithi043 ceph-mon[30062]: from='client.34784 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-82308-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]': finished 2024-03-21T12:54:41.040 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: Running main() from gmock_main.cc 2024-03-21T12:54:41.040 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-03-21T12:54:41.040 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [----------] Global test environment set-up. 2024-03-21T12:54:41.040 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-03-21T12:54:41.040 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-03-21T12:54:41.040 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (63 ms) 2024-03-21T12:54:41.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-03-21T12:54:41.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (58 ms) 2024-03-21T12:54:41.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-03-21T12:54:41.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (14017 ms) 2024-03-21T12:54:41.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-03-21T12:54:41.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:38.056162+0000 mon.a [INF] from='client.25114 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:41.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:38.056334+0000 mon.a [INF] from='client.35132 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-81636-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:38.056638+0000 mon.a [INF] from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:41.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:38.091860+0000 mon.a [INF] from='client.34784 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi043-82308-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]': finished 2024-03-21T12:54:41.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:38.097802+0000 mon.c [INF] from='client.? v1:172.21.15.43:0/3603586041' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:38.098845+0000 mon.c [INF] from='client.? v1:172.21.15.43:0/1457785255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-82758-1"}]: dispatch 2024-03-21T12:54:41.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:38.378352+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/3449806415' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:41.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:38.691896+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-03-21T12:54:41.041 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:38.692534+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-03-21T12:54:41.042 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:38.692951+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-03-21T12:54:41.042 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:38.693356+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-03-21T12:54:41.042 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:38.693756+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-03-21T12:54:41.042 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:38.694222+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-03-21T12:54:41.042 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:38.694725+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-03-21T12:54:41.042 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:38.695116+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-03-21T12:54:41.042 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:38.695495+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-03-21T12:54:41.042 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:38.695893+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-03-21T12:54:41.045 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.144243+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/247888446' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:41.045 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.901369+0000 mon.a [WRN] Health check update: 13 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:54:41.045 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.906769+0000 mon.a [INF] from='client.35204 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.045 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.910120+0000 mon.a [INF] from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-82758-1"}]: dispatch 2024-03-21T12:54:41.045 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.911101+0000 mon.a [INF] from='client.35132 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-81636-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:41.045 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.920404+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/1435054395' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81661-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.045 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.920446+0000 mon.c [INF] from='client.? v1:172.21.15.43:0/1191142118' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.045 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.920743+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/524839068' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.045 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.952432+0000 mon.a [INF] from='client.35246 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.046 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.960646+0000 mon.a [INF] from='client.25114 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:41.046 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.960963+0000 mon.a [INF] from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:41.046 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.961509+0000 mon.a [INF] from='client.25111 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81661-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.046 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.962441+0000 mon.a [INF] from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.046 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.964216+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/247888446' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.046 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.964502+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/3449806415' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-81674-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.046 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.965871+0000 mon.a [INF] from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.046 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.967534+0000 mon.a [INF] from='client.25114 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-81674-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.046 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.975620+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/4261994065' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:41.046 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.982902+0000 mon.a [INF] from='client.25240 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:41.046 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.984425+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/4261994065' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:41.046 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.986353+0000 mon.c [INF] from='client.? v1:172.21.15.43:0/1214900559' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:41.046 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.987099+0000 mon.a [INF] from='client.25240 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:41.046 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.987475+0000 mon.a [INF] from='client.35270 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:41.046 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.988846+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/4261994065' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-82121-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.047 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.989817+0000 mon.a [INF] from='client.25240 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-82121-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.047 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.996085+0000 mon.c [INF] from='client.? v1:172.21.15.43:0/1214900559' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:41.047 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:39.998666+0000 mon.a [INF] from='client.35270 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:41.047 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:40.001801+0000 client.admin [INF] onexx 2024-03-21T12:54:41.047 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:40.003472+0000 mon.c [INF] from='client.? v1:172.21.15.43:0/1214900559' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.047 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:40.003851+0000 mon.a [INF] from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-21T12:54:41.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3603586041' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1457785255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-82758-1"}]: dispatch 2024-03-21T12:54:41.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:54:41.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3449806415' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:41.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-03-21T12:54:41.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-03-21T12:54:41.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-03-21T12:54:41.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-03-21T12:54:41.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-03-21T12:54:41.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-03-21T12:54:41.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-03-21T12:54:41.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-03-21T12:54:41.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-03-21T12:54:41.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-03-21T12:54:41.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-03-21T12:54:41.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-03-21T12:54:41.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-03-21T12:54:41.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-03-21T12:54:41.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-03-21T12:54:41.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-03-21T12:54:41.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-03-21T12:54:41.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-03-21T12:54:41.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-03-21T12:54:41.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-03-21T12:54:41.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:41.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: 18.7 deep-scrub starts 2024-03-21T12:54:41.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: 18.7 deep-scrub ok 2024-03-21T12:54:41.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: pgmap v80: 980 pgs: 40 creating+peering, 836 active+clean, 104 unknown; 624 KiB data, 276 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 159 KiB/s wr, 705 op/s 2024-03-21T12:54:41.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/247888446' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:41.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: 18.6 deep-scrub starts 2024-03-21T12:54:41.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: 18.6 deep-scrub ok 2024-03-21T12:54:41.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: 18.3 deep-scrub starts 2024-03-21T12:54:41.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: 18.3 deep-scrub ok 2024-03-21T12:54:41.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:41.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: osdmap e79: 8 total, 8 up, 8 in 2024-03-21T12:54:41.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: Health check update: 13 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:54:41.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.35204 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-82758-1"}]: dispatch 2024-03-21T12:54:41.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.35132 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-81636-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:41.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: osdmap e80: 8 total, 8 up, 8 in 2024-03-21T12:54:41.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1435054395' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81661-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1191142118' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/524839068' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.461 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.35246 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.461 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.25114 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:41.461 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:41.461 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.25111 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81661-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.461 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.461 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/247888446' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.461 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3449806415' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-81674-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.461 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.461 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.25114 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-81674-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.461 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4261994065' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:41.461 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.25240 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:41.461 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4261994065' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:41.461 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1214900559' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:41.461 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.25240 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:41.461 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.35270 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:41.462 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4261994065' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-82121-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.462 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.25240 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-82121-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.462 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1214900559' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:41.462 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.35270 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:41.462 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: onexx 2024-03-21T12:54:41.462 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1214900559' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.462 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-21T12:54:41.462 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.35270 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.462 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:41.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3603586041' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1457785255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-82758-1"}]: dispatch 2024-03-21T12:54:41.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:54:41.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3449806415' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:41.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-03-21T12:54:41.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-03-21T12:54:41.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-03-21T12:54:41.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-03-21T12:54:41.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-03-21T12:54:41.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-03-21T12:54:41.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-03-21T12:54:41.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-03-21T12:54:41.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-03-21T12:54:41.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-03-21T12:54:41.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-03-21T12:54:41.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-03-21T12:54:41.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-03-21T12:54:41.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-03-21T12:54:41.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-03-21T12:54:41.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-03-21T12:54:41.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-03-21T12:54:41.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-03-21T12:54:41.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-03-21T12:54:41.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3603586041' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1457785255' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-82758-1"}]: dispatch 2024-03-21T12:54:41.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:54:41.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3449806415' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:41.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-03-21T12:54:41.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-03-21T12:54:41.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-03-21T12:54:41.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-03-21T12:54:41.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-03-21T12:54:41.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-03-21T12:54:41.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-03-21T12:54:41.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-03-21T12:54:41.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-03-21T12:54:41.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-03-21T12:54:41.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-03-21T12:54:41.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-03-21T12:54:41.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-03-21T12:54:41.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-03-21T12:54:41.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-03-21T12:54:41.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-03-21T12:54:41.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-03-21T12:54:41.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-03-21T12:54:41.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-03-21T12:54:41.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-03-21T12:54:41.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:41.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: 18.7 deep-scrub starts 2024-03-21T12:54:41.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: 18.7 deep-scrub ok 2024-03-21T12:54:41.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: pgmap v80: 980 pgs: 40 creating+peering, 836 active+clean, 104 unknown; 624 KiB data, 276 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 159 KiB/s wr, 705 op/s 2024-03-21T12:54:41.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/247888446' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:41.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: 18.6 deep-scrub starts 2024-03-21T12:54:41.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: 18.6 deep-scrub ok 2024-03-21T12:54:41.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: 18.3 deep-scrub starts 2024-03-21T12:54:41.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: 18.3 deep-scrub ok 2024-03-21T12:54:41.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:41.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: osdmap e79: 8 total, 8 up, 8 in 2024-03-21T12:54:41.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: Health check update: 13 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:54:41.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.35204 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-82758-1"}]: dispatch 2024-03-21T12:54:41.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.35132 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-81636-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:41.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: osdmap e80: 8 total, 8 up, 8 in 2024-03-21T12:54:41.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1435054395' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81661-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1191142118' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/524839068' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.35246 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.25114 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:41.489 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:41.490 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.25111 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81661-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.490 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.490 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-03-21T12:54:41.490 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:41.490 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: 18.7 deep-scrub starts 2024-03-21T12:54:41.490 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: 18.7 deep-scrub ok 2024-03-21T12:54:41.490 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: pgmap v80: 980 pgs: 40 creating+peering, 836 active+clean, 104 unknown; 624 KiB data, 276 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 159 KiB/s wr, 705 op/s 2024-03-21T12:54:41.490 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/247888446' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:41.490 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: 18.6 deep-scrub starts 2024-03-21T12:54:41.490 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: 18.6 deep-scrub ok 2024-03-21T12:54:41.490 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: 18.3 deep-scrub starts 2024-03-21T12:54:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: 18.3 deep-scrub ok 2024-03-21T12:54:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: osdmap e79: 8 total, 8 up, 8 in 2024-03-21T12:54:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: Health check update: 13 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:54:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.35204 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-82758-1"}]: dispatch 2024-03-21T12:54:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.35132 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi043-81636-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: osdmap e80: 8 total, 8 up, 8 in 2024-03-21T12:54:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1435054395' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81661-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1191142118' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/524839068' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.35246 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.25114 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:41.491 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.25111 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81661-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.492 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/247888446' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3449806415' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-81674-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.25114 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-81674-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4261994065' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.25240 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4261994065' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1214900559' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.25240 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:41.492 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.35270 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:41.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4261994065' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-82121-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.25240 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-82121-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1214900559' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:41.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.35270 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:41.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: onexx 2024-03-21T12:54:41.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1214900559' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-21T12:54:41.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.35270 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.493 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:41.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/247888446' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3449806415' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-81674-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.493 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.25114 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-81674-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4261994065' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:41.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.25240 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:41.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4261994065' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:41.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1214900559' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:41.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.25240 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:41.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.35270 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:41.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4261994065' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-82121-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.25240 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-82121-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1214900559' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:41.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.35270 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:41.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: onexx 2024-03-21T12:54:41.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1214900559' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-21T12:54:41.494 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.35270 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:41.495 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:42.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: 18.9 deep-scrub starts 2024-03-21T12:54:42.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: 18.9 deep-scrub ok 2024-03-21T12:54:42.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: 18.8 deep-scrub starts 2024-03-21T12:54:42.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: 18.8 deep-scrub ok 2024-03-21T12:54:42.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: 18.2 deep-scrub starts 2024-03-21T12:54:42.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: 18.2 deep-scrub ok 2024-03-21T12:54:42.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: 18.5 deep-scrub starts 2024-03-21T12:54:42.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: 18.5 deep-scrub ok 2024-03-21T12:54:42.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-03-21T12:54:42.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: from='client.35204 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:42.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-82758-1"}]': finished 2024-03-21T12:54:42.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: from='client.35246 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:42.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81661-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:42.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: from='client.25075 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:42.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:42.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: from='client.25114 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-81674-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:42.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: from='client.25240 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-82121-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:42.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: from='client.35270 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:42.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1457785255' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-82758-1"}]: dispatch 2024-03-21T12:54:42.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3449806415' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-81674-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:42.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/247888446' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-81695-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:42.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4261994065' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-82121-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:42.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: osdmap e81: 8 total, 8 up, 8 in 2024-03-21T12:54:42.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1214900559' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-82055-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:42.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: from='client.25114 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-81674-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:42.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-81695-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:42.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: from='client.25240 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-82121-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:42.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3458186974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:42.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/282144841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:42.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-82758-1"}]: dispatch 2024-03-21T12:54:42.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: from='client.35270 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-82055-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:42.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:42.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: 18.9 deep-scrub starts 2024-03-21T12:54:42.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: 18.9 deep-scrub ok 2024-03-21T12:54:42.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: 18.8 deep-scrub starts 2024-03-21T12:54:42.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: 18.8 deep-scrub ok 2024-03-21T12:54:42.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: 18.2 deep-scrub starts 2024-03-21T12:54:42.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: 18.2 deep-scrub ok 2024-03-21T12:54:42.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: 18.5 deep-scrub starts 2024-03-21T12:54:42.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: 18.5 deep-scrub ok 2024-03-21T12:54:42.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-03-21T12:54:42.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: from='client.35204 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:42.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-82758-1"}]': finished 2024-03-21T12:54:42.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: from='client.35246 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:42.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81661-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:42.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: from='client.25075 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:42.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:42.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: from='client.25114 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-81674-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:42.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: from='client.25240 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-82121-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:42.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: from='client.35270 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:42.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1457785255' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-82758-1"}]: dispatch 2024-03-21T12:54:42.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3449806415' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-81674-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:42.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/247888446' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-81695-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:42.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4261994065' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-82121-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:42.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: osdmap e81: 8 total, 8 up, 8 in 2024-03-21T12:54:42.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: 18.9 deep-scrub starts 2024-03-21T12:54:42.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: 18.9 deep-scrub ok 2024-03-21T12:54:42.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: 18.8 deep-scrub starts 2024-03-21T12:54:42.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: 18.8 deep-scrub ok 2024-03-21T12:54:42.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: 18.2 deep-scrub starts 2024-03-21T12:54:42.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: 18.2 deep-scrub ok 2024-03-21T12:54:42.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: 18.5 deep-scrub starts 2024-03-21T12:54:42.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: 18.5 deep-scrub ok 2024-03-21T12:54:42.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-03-21T12:54:42.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: from='client.35204 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:42.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi043-82758-1"}]': finished 2024-03-21T12:54:42.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: from='client.35246 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:42.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81661-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:42.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: from='client.25075 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:42.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:42.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: from='client.25114 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi043-81674-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:42.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: from='client.25240 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi043-82121-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:42.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: from='client.35270 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:42.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1457785255' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-82758-1"}]: dispatch 2024-03-21T12:54:42.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3449806415' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-81674-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:42.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/247888446' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-81695-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:42.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4261994065' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-82121-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:42.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: osdmap e81: 8 total, 8 up, 8 in 2024-03-21T12:54:42.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1214900559' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-82055-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:42.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: from='client.25114 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-81674-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:42.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-81695-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:42.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: from='client.25240 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-82121-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:42.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3458186974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:42.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/282144841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:42.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-82758-1"}]: dispatch 2024-03-21T12:54:42.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: from='client.35270 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-82055-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:42.488 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:42.489 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1214900559' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-82055-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:42.489 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: from='client.25114 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-81674-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:42.489 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-81695-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:42.489 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: from='client.25240 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-82121-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:42.489 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3458186974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:42.489 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/282144841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:42.489 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: from='client.34886 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-82758-1"}]: dispatch 2024-03-21T12:54:42.489 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: from='client.35270 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-82055-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:42.489 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:43.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:43 smithi155 ceph-mon[33846]: twoxx 2024-03-21T12:54:43.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-03-21T12:54:43.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:43 smithi155 ceph-mon[33846]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:43.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:43 smithi155 ceph-mon[33846]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-82758-1"}]': finished 2024-03-21T12:54:43.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1759292229' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-82926-1"}]: dispatch 2024-03-21T12:54:43.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4010074928' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-82758-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:43.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:43.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/524839068' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:43.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1435054395' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81661-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:43.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:43 smithi155 ceph-mon[33846]: osdmap e82: 8 total, 8 up, 8 in 2024-03-21T12:54:43.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:43 smithi043 ceph-mon[30062]: twoxx 2024-03-21T12:54:43.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-03-21T12:54:43.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:43 smithi043 ceph-mon[30062]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:43.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:43 smithi043 ceph-mon[30062]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-82758-1"}]': finished 2024-03-21T12:54:43.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1759292229' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-82926-1"}]: dispatch 2024-03-21T12:54:43.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4010074928' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-82758-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:43.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:43.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/524839068' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:43.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1435054395' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81661-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:43.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:43 smithi043 ceph-mon[30062]: osdmap e82: 8 total, 8 up, 8 in 2024-03-21T12:54:43.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:43 smithi043 ceph-mon[41699]: twoxx 2024-03-21T12:54:43.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-03-21T12:54:43.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:43 smithi043 ceph-mon[41699]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:43.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:43 smithi043 ceph-mon[41699]: from='client.34886 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi043-82758-1"}]': finished 2024-03-21T12:54:43.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1759292229' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-82926-1"}]: dispatch 2024-03-21T12:54:43.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4010074928' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-82758-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:43.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:43.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/524839068' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:43.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1435054395' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81661-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:43.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:43 smithi043 ceph-mon[41699]: osdmap e82: 8 total, 8 up, 8 in 2024-03-21T12:54:44.222 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:40.004433+0000 mon.a [INF] from='client.35270 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name delete_pools_parallel: process_1_[82628]: starting. 2024-03-21T12:54:44.222 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_1_[82628]: creating pool ceph_test_rados_delete_pools_parallel.smithi043-82544 2024-03-21T12:54:44.222 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_1_[82628]: created object 0... 2024-03-21T12:54:44.222 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_1_[82628]: created object 25... 2024-03-21T12:54:44.223 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_1_[82628]: created object 49... 2024-03-21T12:54:44.223 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_1_[82628]: finishing. 2024-03-21T12:54:44.223 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_1_[82628]: shutting down. 2024-03-21T12:54:44.223 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_2_[82630]: starting. 2024-03-21T12:54:44.223 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_2_[82630]: deleting pool ceph_test_rados_delete_pools_parallel.smithi043-82544 2024-03-21T12:54:44.223 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_2_[82630]: shutting down. 2024-03-21T12:54:44.223 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: ******************************* 2024-03-21T12:54:44.223 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[84083]: starting. 2024-03-21T12:54:44.223 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[84083]: creating pool ceph_test_rados_delete_pools_parallel.smithi043-82544 2024-03-21T12:54:44.223 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[84083]: created object 0... 2024-03-21T12:54:44.223 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[84083]: created object 25... 2024-03-21T12:54:44.223 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[84083]: created object 49... 2024-03-21T12:54:44.223 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[84083]: finishing. 2024-03-21T12:54:44.224 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_3_[84083]: shutting down. 2024-03-21T12:54:44.224 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: ******************************* 2024-03-21T12:54:44.224 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_5_[84085]: starting. 2024-03-21T12:54:44.224 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_5_[84085]: listing objects. 2024-03-21T12:54:44.224 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_5_[84085]: listed object 0... 2024-03-21T12:54:44.224 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_5_[84085]: listed object 25... 2024-03-21T12:54:44.224 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_5_[84085]: saw 50 objects 2024-03-21T12:54:44.224 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_5_[84085]: shutting down. 2024-03-21T12:54:44.224 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: ******************************* 2024-03-21T12:54:44.224 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_4_[84084]: starting. 2024-03-21T12:54:44.224 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_4_[84084]: deleting pool ceph_test_rados_delete_pools_parallel.smithi043-82544 2024-03-21T12:54:44.224 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: process_4_[84084]: shutting down. 2024-03-21T12:54:44.224 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: ******************************* 2024-03-21T12:54:44.224 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: ******************************* 2024-03-21T12:54:44.224 INFO:tasks.workunit.client.0.smithi043.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-03-21T12:54:44.225 INFO:tasks.workunit.client.0.smithi043.stdout: cls: Running main() from gmock_main.cc 2024-03-21T12:54:44.225 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [==========] Running 2 tests from 1 test suite. 2024-03-21T12:54:44.225 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [----------] Global test environment set-up. 2024-03-21T12:54:44.225 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [----------] 2 tests from NeoRadosCls 2024-03-21T12:54:44.225 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [ RUN ] NeoRadosCls.DNE 2024-03-21T12:54:44.226 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [ OK ] NeoRadosCls.DNE (11330 ms) 2024-03-21T12:54:44.226 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [ RUN ] NeoRadosCls.RemoteReads 2024-03-21T12:54:44.226 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [ OK ] NeoRadosCls.RemoteReads (6980 ms) 2024-03-21T12:54:44.226 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [----------] 2 tests from NeoRadosCls (18310 ms total) 2024-03-21T12:54:44.226 INFO:tasks.workunit.client.0.smithi043.stdout: cls: 2024-03-21T12:54:44.226 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [----------] Global test environment tear-down 2024-03-21T12:54:44.226 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [==========] 2 tests from 1 test suite ran. (18310 ms total) 2024-03-21T12:54:44.226 INFO:tasks.workunit.client.0.smithi043.stdout: cls: [ PASSED ] 2 tests. 2024-03-21T12:54:44.240 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[82610]: starting. 2024-03-21T12:54:44.240 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[82610]: creating pool ceph_test_rados_open_pools_parallel.smithi043-82501 2024-03-21T12:54:44.240 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[82610]: created object 0... 2024-03-21T12:54:44.240 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[82610]: created object 25... 2024-03-21T12:54:44.240 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[82610]: created object 49... 2024-03-21T12:54:44.240 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[82610]: finishing. 2024-03-21T12:54:44.240 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_1_[82610]: shutting down. 2024-03-21T12:54:44.240 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_2_[82611]: starting. 2024-03-21T12:54:44.240 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_2_[82611]: rados_pool_create. 2024-03-21T12:54:44.240 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_2_[82611]: rados_ioctx_create. 2024-03-21T12:54:44.240 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_2_[82611]: shutting down. 2024-03-21T12:54:44.240 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: ******************************* 2024-03-21T12:54:44.241 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[84078]: starting. 2024-03-21T12:54:44.241 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[84078]: creating pool ceph_test_rados_open_pools_parallel.smithi043-82501 2024-03-21T12:54:44.241 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[84078]: created object 0... 2024-03-21T12:54:44.241 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[84078]: created object 25... 2024-03-21T12:54:44.241 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[84078]: created object 49... 2024-03-21T12:54:44.241 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[84078]: finishing. 2024-03-21T12:54:44.241 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_3_[84078]: shutting down. 2024-03-21T12:54:44.241 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: ******************************* 2024-03-21T12:54:44.241 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_4_[84079]: starting. 2024-03-21T12:54:44.241 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_4_[84079]: rados_pool_create. 2024-03-21T12:54:44.241 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_4_[84079]: rados_ioctx_create. 2024-03-21T12:54:44.241 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: process_4_[84079]: shutting down. 2024-03-21T12:54:44.241 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: ******************************* 2024-03-21T12:54:44.241 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: ******************************* 2024-03-21T12:54:44.241 INFO:tasks.workunit.client.0.smithi043.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-03-21T12:54:44.429 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: Running main() from gmock_main.cc 2024-03-21T12:54:44.429 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2024-03-21T12:54:44.429 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [----------] Global test environment set-up. 2024-03-21T12:54:44.429 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2024-03-21T12:54:44.430 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: seed 81695 2024-03-21T12:54:44.430 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-03-21T12:54:44.430 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-03-21T12:54:44.430 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-03-21T12:54:44.430 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (92 ms) 2024-03-21T12:54:44.430 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-03-21T12:54:44.430 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (10 ms) 2024-03-21T12:54:44.430 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-03-21T12:54:44.430 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (2 ms) 2024-03-21T12:54:44.430 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-03-21T12:54:44.430 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (2 ms) 2024-03-21T12:54:44.430 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-03-21T12:54:44.430 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (5 ms) 2024-03-21T12:54:44.430 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-03-21T12:54:44.431 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (2 ms) 2024-03-21T12:54:44.431 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-03-21T12:54:44.431 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (1 ms) 2024-03-21T12:54:44.431 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-03-21T12:54:44.431 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (5 ms) 2024-03-21T12:54:44.431 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-03-21T12:54:44.431 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (3 ms) 2024-03-21T12:54:44.431 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-03-21T12:54:44.431 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (4 ms) 2024-03-21T12:54:44.431 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-03-21T12:54:44.431 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (2 ms) 2024-03-21T12:54:44.431 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-03-21T12:54:44.431 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (3 ms) 2024-03-21T12:54:44.431 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-03-21T12:54:44.432 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (2 ms) 2024-03-21T12:54:44.432 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-03-21T12:54:44.432 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (3 ms) 2024-03-21T12:54:44.432 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-03-21T12:54:44.432 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (7 ms) 2024-03-21T12:54:44.432 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-03-21T12:54:44.432 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (4 ms) 2024-03-21T12:54:44.432 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-03-21T12:54:44.432 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (2 ms) 2024-03-21T12:54:44.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-03-21T12:54:44.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (2 ms) 2024-03-21T12:54:44.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-03-21T12:54:44.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (2 ms) 2024-03-21T12:54:44.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (153 ms total) 2024-03-21T12:54:44.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: 2024-03-21T12:54:44.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-03-21T12:54:44.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-03-21T12:54:44.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (1198 ms) 2024-03-21T12:54:44.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-03-21T12:54:44.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (18 ms) 2024-03-21T12:54:44.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-03-21T12:54:44.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (3 ms) 2024-03-21T12:54:44.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-03-21T12:54:44.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (2 ms) 2024-03-21T12:54:44.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-03-21T12:54:44.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (3 ms) 2024-03-21T12:54:44.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: 18.1 deep-scrub starts 2024-03-21T12:54:44.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: 18.1 deep-scrub ok 2024-03-21T12:54:44.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: 18.0 deep-scrub starts 2024-03-21T12:54:44.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: 18.0 deep-scrub ok 2024-03-21T12:54:44.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: pgmap v83: 884 pgs: 40 creating+peering, 484 active+clean, 360 unknown; 587 KiB data, 276 MiB used, 715 GiB / 715 GiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 519 op/s 2024-03-21T12:54:44.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: 18.4 deep-scrub starts 2024-03-21T12:54:44.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: 18.4 deep-scrub ok 2024-03-21T12:54:44.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: pgmap v85: 940 pgs: 130 creating+peering, 794 active+clean, 16 unknown; 145 MiB data, 729 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:54:44.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3458186974' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:44.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-03-21T12:54:44.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: from='client.25114 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-81674-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-81674-16"}]': finished 2024-03-21T12:54:44.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-81695-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-81695-22"}]': finished 2024-03-21T12:54:44.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: from='client.25240 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-82121-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-82121-7"}]': finished 2024-03-21T12:54:44.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: from='client.35270 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-82055-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-82055-7"}]': finished 2024-03-21T12:54:44.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: osdmap e83: 8 total, 8 up, 8 in 2024-03-21T12:54:44.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: from='client.25312 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:44.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/282144841' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:44.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-82758-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:44.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: from='client.24886 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-82926-1"}]: dispatch 2024-03-21T12:54:44.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:44.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3458186974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:44.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:44.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: from='client.25111 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81661-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:44.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:44.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: from='client.25312 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:44.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:44.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/282144841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-81775-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:44.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:44 smithi155 ceph-mon[33846]: from='client.25312 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-81775-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:44.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: 18.1 deep-scrub starts 2024-03-21T12:54:44.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: 18.1 deep-scrub ok 2024-03-21T12:54:44.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: 18.0 deep-scrub starts 2024-03-21T12:54:44.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: 18.0 deep-scrub ok 2024-03-21T12:54:44.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: pgmap v83: 884 pgs: 40 creating+peering, 484 active+clean, 360 unknown; 587 KiB data, 276 MiB used, 715 GiB / 715 GiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 519 op/s 2024-03-21T12:54:44.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: 18.4 deep-scrub starts 2024-03-21T12:54:44.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: 18.4 deep-scrub ok 2024-03-21T12:54:44.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: pgmap v85: 940 pgs: 130 creating+peering, 794 active+clean, 16 unknown; 145 MiB data, 729 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:54:44.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3458186974' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:44.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-03-21T12:54:44.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: from='client.25114 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-81674-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-81674-16"}]': finished 2024-03-21T12:54:44.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-81695-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-81695-22"}]': finished 2024-03-21T12:54:44.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: from='client.25240 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-82121-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-82121-7"}]': finished 2024-03-21T12:54:44.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: from='client.35270 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-82055-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-82055-7"}]': finished 2024-03-21T12:54:44.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: osdmap e83: 8 total, 8 up, 8 in 2024-03-21T12:54:44.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: from='client.25312 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:44.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/282144841' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:44.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-82758-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:44.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: from='client.24886 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-82926-1"}]: dispatch 2024-03-21T12:54:44.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:44.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3458186974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:44.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:44.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: from='client.25111 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81661-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:44.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:44.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: from='client.25312 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:44.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:44.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/282144841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-81775-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:44.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[30062]: from='client.25312 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-81775-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:44.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: 18.1 deep-scrub starts 2024-03-21T12:54:44.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: 18.1 deep-scrub ok 2024-03-21T12:54:44.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: 18.0 deep-scrub starts 2024-03-21T12:54:44.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: 18.0 deep-scrub ok 2024-03-21T12:54:44.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: pgmap v83: 884 pgs: 40 creating+peering, 484 active+clean, 360 unknown; 587 KiB data, 276 MiB used, 715 GiB / 715 GiB avail; 3.8 KiB/s rd, 22 KiB/s wr, 519 op/s 2024-03-21T12:54:44.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: 18.4 deep-scrub starts 2024-03-21T12:54:44.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: 18.4 deep-scrub ok 2024-03-21T12:54:44.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: pgmap v85: 940 pgs: 130 creating+peering, 794 active+clean, 16 unknown; 145 MiB data, 729 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:54:44.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3458186974' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:44.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-03-21T12:54:44.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: from='client.25114 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-81674-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-81674-16"}]': finished 2024-03-21T12:54:44.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-81695-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-81695-22"}]': finished 2024-03-21T12:54:44.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: from='client.25240 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-82121-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-82121-7"}]': finished 2024-03-21T12:54:44.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: from='client.35270 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-82055-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-82055-7"}]': finished 2024-03-21T12:54:44.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: osdmap e83: 8 total, 8 up, 8 in 2024-03-21T12:54:44.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: from='client.25312 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:44.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/282144841' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:44.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-82758-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:44.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: from='client.24886 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-82926-1"}]: dispatch 2024-03-21T12:54:44.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:44.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3458186974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:44.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:44.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: from='client.25111 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81661-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:44.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:44.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: from='client.25312 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:44.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:44.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/282144841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-81775-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:44.488 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:44 smithi043 ceph-mon[41699]: from='client.25312 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-81775-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:45.301 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] L": "testprofile-LibRadosStatECPP_smithi043-82055-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:45.301 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:43.187517+0000 mon.c [INF] from='client.? v1:172.21.15.43:0/3458186974' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:45.301 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:43.193322+0000 mon.a [INF] from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-03-21T12:54:45.301 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:43.199445+0000 mon.a [INF] from='client.25114 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi043-81674-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi043-81674-16"}]': finished 2024-03-21T12:54:45.302 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:43.199556+0000 mon.a [INF] from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi043-81695-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi043-81695-22"}]': finished 2024-03-21T12:54:45.302 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:43.199636+0000 mon.a [INF] from='client.25240 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi043-82121-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi043-82121-7"}]': finished 2024-03-21T12:54:45.302 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:43.199713+0000 mon.a [INF] from='client.35270 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi043-82055-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi043-82055-7"}]': finished 2024-03-21T12:54:45.302 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:43.245105+0000 mon.a [INF] from='client.25312 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:45.302 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:43.246932+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/282144841' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:45.302 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:43.510261+0000 mon.a [INF] from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-82758-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:45.302 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:43.764596+0000 mon.a [INF] from='client.24886 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-82926-1"}]: dispatch 2024-03-21T12:54:45.302 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:43.779994+0000 mon.a [INF] from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:45.302 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:43.780556+0000 mon.c [INF] from='client.? v1:172.21.15.43:0/3458186974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:45.302 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:43.782755+0000 mon.a [INF] from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:45.302 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:43.783092+0000 mon.a [INF] from='client.25111 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81661-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:46.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.051422+0000 mon.a [INF] from='client.25312 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:46.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.052038+0000 mon.a [INF] from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:46.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.052086+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/282144841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-81775-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:46.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.052707+0000 mon.a [INF] from='client.25312 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-81775-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:46.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.203639+0000 mon.a [INF] from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-82758-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:46.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.203744+0000 mon.a [INF] from='client.24886 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-82926-1"}]': finished 2024-03-21T12:54:46.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.203812+0000 mon.a [INF] from='client.25075 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:46.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.203855+0000 mon.a [INF] from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81661-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:46.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.203923+0000 mon.a [INF] from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:46.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.203981+0000 mon.a [INF] from='client.25312 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-81775-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:46.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.208278+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/1759292229' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-82926-1"}]: dispatch 2024-03-21T12:54:46.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.208615+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/282144841' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-81775-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:46.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.209688+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/1435054395' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-81661-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-21T12:54:46.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.209916+0000 mon.a [INF] from='client.24886 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-82926-1"}]: dispatch 2024-03-21T12:54:46.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.210902+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/524839068' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-4", "tierpool": "test-rados-api-smithi043-81880-4-cache"}]: dispatch 2024-03-21T12:54:46.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-82758-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:46.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: from='client.24886 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-82926-1"}]': finished 2024-03-21T12:54:46.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: from='client.25075 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:46.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81661-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:46.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:46.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: from='client.25312 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-81775-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:46.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: osdmap e84: 8 total, 8 up, 8 in 2024-03-21T12:54:46.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1759292229' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-82926-1"}]: dispatch 2024-03-21T12:54:46.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/282144841' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-81775-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:46.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1435054395' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-81661-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-21T12:54:46.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: from='client.24886 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-82926-1"}]: dispatch 2024-03-21T12:54:46.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/524839068' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-4", "tierpool": "test-rados-api-smithi043-81880-4-cache"}]: dispatch 2024-03-21T12:54:46.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: from='client.25312 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-81775-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:46.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: from='client.25111 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-81661-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-21T12:54:46.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3458186974' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-81755-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:46.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4010074928' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-82758-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-82758-2"}]: dispatch 2024-03-21T12:54:46.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2542191100' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:46.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4042212061' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-81636-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:46.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/453188979' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:46.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-4", "tierpool": "test-rados-api-smithi043-81880-4-cache"}]: dispatch 2024-03-21T12:54:46.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-81755-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:46.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-82758-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-82758-2"}]: dispatch 2024-03-21T12:54:46.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: from='client.34784 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:46.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: from='client.35336 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-81636-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:46.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: from='client.35342 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:46.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:46.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-82758-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:46.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: from='client.24886 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-82926-1"}]': finished 2024-03-21T12:54:46.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: from='client.25075 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:46.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81661-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:46.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:46.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: from='client.25312 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-81775-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:46.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: osdmap e84: 8 total, 8 up, 8 in 2024-03-21T12:54:46.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1759292229' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-82926-1"}]: dispatch 2024-03-21T12:54:46.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/282144841' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-81775-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:46.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1435054395' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-81661-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-21T12:54:46.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: from='client.24886 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-82926-1"}]: dispatch 2024-03-21T12:54:46.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/524839068' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-4", "tierpool": "test-rados-api-smithi043-81880-4-cache"}]: dispatch 2024-03-21T12:54:46.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: from='client.25312 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-81775-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:46.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: from='client.25111 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-81661-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-21T12:54:46.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3458186974' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-81755-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:46.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4010074928' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-82758-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-82758-2"}]: dispatch 2024-03-21T12:54:46.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2542191100' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:46.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4042212061' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-81636-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:46.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/453188979' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:46.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-4", "tierpool": "test-rados-api-smithi043-81880-4-cache"}]: dispatch 2024-03-21T12:54:46.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-81755-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:46.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-82758-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-82758-2"}]: dispatch 2024-03-21T12:54:46.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: from='client.34784 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:46.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: from='client.35336 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-81636-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:46.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: from='client.35342 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:46.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:46.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi043-82758-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:46.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: from='client.24886 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi043-82926-1"}]': finished 2024-03-21T12:54:46.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: from='client.25075 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:46.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi043-81661-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:46.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:46.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: from='client.25312 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi043-81775-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:46.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: osdmap e84: 8 total, 8 up, 8 in 2024-03-21T12:54:46.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1759292229' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-82926-1"}]: dispatch 2024-03-21T12:54:46.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/282144841' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-81775-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:46.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1435054395' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-81661-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-21T12:54:46.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: from='client.24886 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-82926-1"}]: dispatch 2024-03-21T12:54:46.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/524839068' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-4", "tierpool": "test-rados-api-smithi043-81880-4-cache"}]: dispatch 2024-03-21T12:54:46.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: from='client.25312 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-81775-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:46.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: from='client.25111 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-81661-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-21T12:54:46.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3458186974' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-81755-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:46.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4010074928' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-82758-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-82758-2"}]: dispatch 2024-03-21T12:54:46.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2542191100' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:46.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4042212061' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-81636-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:46.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/453188979' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:46.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-4", "tierpool": "test-rados-api-smithi043-81880-4-cache"}]: dispatch 2024-03-21T12:54:46.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-81755-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:46.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-82758-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-82758-2"}]: dispatch 2024-03-21T12:54:46.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: from='client.34784 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:46.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: from='client.35336 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-81636-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:46.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: from='client.35342 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:46.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:47.141 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.213337+0000 mon.a [INF] from='client.25312 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smith cmd: Running main() from gmock_main.cc 2024-03-21T12:54:47.141 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [==========] Running 3 tests from 1 test suite. 2024-03-21T12:54:47.141 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [----------] Global test environment set-up. 2024-03-21T12:54:47.141 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [----------] 3 tests from NeoRadosCmd 2024-03-21T12:54:47.141 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [ RUN ] NeoRadosCmd.MonDescribe 2024-03-21T12:54:47.141 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [ OK ] NeoRadosCmd.MonDescribe (10264 ms) 2024-03-21T12:54:47.142 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [ RUN ] NeoRadosCmd.OSDCmd 2024-03-21T12:54:47.142 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [ OK ] NeoRadosCmd.OSDCmd (4847 ms) 2024-03-21T12:54:47.142 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [ RUN ] NeoRadosCmd.PGCmd 2024-03-21T12:54:47.142 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [ OK ] NeoRadosCmd.PGCmd (6079 ms) 2024-03-21T12:54:47.142 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [----------] 3 tests from NeoRadosCmd (21190 ms total) 2024-03-21T12:54:47.142 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: 2024-03-21T12:54:47.142 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [----------] Global test environment tear-down 2024-03-21T12:54:47.142 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [==========] 3 tests from 1 test suite ran. (21191 ms total) 2024-03-21T12:54:47.142 INFO:tasks.workunit.client.0.smithi043.stdout: cmd: [ PASSED ] 3 tests. 2024-03-21T12:54:47.771 INFO:tasks.workunit.client.0.smithi043.stdout:i043-81775-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:47.771 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.213735+0000 mon.a [INF] from='client.25111 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-81661-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-21T12:54:47.772 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.219761+0000 mon.c [INF] from='client.? v1:172.21.15.43:0/3458186974' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-81755-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:47.772 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.219949+0000 mon.c [INF] from='client.? v1:172.21.15.43:0/4010074928' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-82758-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-82758-2"}]: dispatch 2024-03-21T12:54:47.772 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.227230+0000 mon.c [INF] from='client.? v1:172.21.15.43:0/2542191100' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:47.772 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.230615+0000 mon.c [INF] from='client.? v1:172.21.15.43:0/4042212061' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-81636-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:47.772 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.231458+0000 mon.c [INF] from='client.? v1:172.21.15.43:0/453188979' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:47.772 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.244438+0000 mon.a [INF] from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-4", "tierpool": "test-rados-api-smithi043-81880-4-cache"}]: dispatch 2024-03-21T12:54:47.772 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.244980+0000 mon.a [INF] from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-81755-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:47.772 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.245791+0000 mon.a [INF] from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-82758-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-82758-2"}]: dispatch 2024-03-21T12:54:47.772 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.246878+0000 mon.a [INF] from='client.34784 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:47.772 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.247165+0000 mon.a [INF] from='client.35336 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-81636-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:47.772 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:44.247541+0000 mon.a [INF] from='client.35342 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:47.772 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:45.193835+0000 client.admin [INF] threexx 2024-03-21T12:54:47.772 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:45.302284+0000 mon.a [INF] from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-21T12:54:47.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:45.303145+0000 mon.a [INF] from='client.24886 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-82926-1"}]': finished 2024-03-21T12:54:47.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:45.303254+0000 mon.a [INF] from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-81661-3", "field": "max_bytes", "val": "4096"}]': finished 2024-03-21T12:54:47.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:45.303348+0000 mon.a [INF] from='client.25075 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-4", "tierpool": "test-rados-api-smithi043-81880-4-cache"}]': finished 2024-03-21T12:54:47.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:45.303429+0000 mon.a [INF] from='client.34784 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]': finished 2024-03-21T12:54:47.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:45.303514+0000 mon.a [INF] from='client.35336 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-81636-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:47.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:45.303597+0000 mon.a [INF] from='client.35342 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:47.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:45.309844+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/524839068' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-4", "overlaypool": "test-rados-api-smithi043-81880-4-cache"}]: dispatch 2024-03-21T12:54:47.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:45.310559+0000 mon.c [INF] from='client.? v1:172.21.15.43:0/2542191100' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:47.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:45.344874+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/1890594059' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-82926-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:47.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:46.085022+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/4129264064' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:47.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:46.349808+0000 mon.a [INF] from='client.25312 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-81775-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-81775-10"}]': finished 2024-03-21T12:54:47.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:46.349878+0000 mon.a [INF] from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-81755-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-81755-10"}]': finished 2024-03-21T12:54:47.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:46.349957+0000 mon.a [INF] from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-82758-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-82758-2"}]': finished 2024-03-21T12:54:47.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:46.378983+0000 mon.a [INF] from='client.34784 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:47.773 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:46.386367+0000 mon.a [INF] from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-4", "overlaypool": "test-rados-api-smithi043-81880-4-cache"}]: dispatch 2024-03-21T12:54:47.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:46.644789+0000 mon.a [INF] from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-82926-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:47.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:46.647923+0000 mon.a [INF] from='client.25357 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:47.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:47.125452+0000 mon.a [WRN] Health check update: 17 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:54:47.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:47.129758+0000 mon.a [INF] from='client.34784 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82308-1"}]': finished 2024-03-21T12:54:47.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:47.129868+0000 mon.a [INF] from='client.25075 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-4", "overlaypool": "test-rados-api-smithi043-81880-4-cache"}]': finished 2024-03-21T12:54:47.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:47.129974+0000 mon.a [INF] from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-82926-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:47.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:47.130088+0000 mon.a [INF] from='client.25357 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:47.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:47.135623+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/524839068' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-4-cache", "mode": "writeback"}]: dispatch 2024-03-21T12:54:47.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:47.136401+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/1890594059' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-82926-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-82926-2"}]: dispatch 2024-03-21T12:54:47.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:47.137360+0000 mon.a [INF] from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-4-cache", "mode": "writeback"}]: dispatch 2024-03-21T12:54:47.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:47.138510+0000 mon.c [INF] from='client.? v1:172.21.15.43:0/1214900559' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:47.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:47.140624+0000 mon.a [INF] from='client.35270 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:47.774 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:47.141985+0000 mon.a [INF] from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-82926-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-82926-2"}]: dispatch 2024-03-21T12:54:47.775 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:47.148990+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/3449806415' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:47.775 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:47.149304+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/4261994065' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:47.775 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:47.150022+0000 mon.a [INF] from='client.25114 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:48.023 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: pgmap v88: 804 pgs: 112 creating+peering, 548 active+clean, 144 unknown; 145 MiB data, 729 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 6 op/s 2024-03-21T12:54:48.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: threexx 2024-03-21T12:54:48.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-21T12:54:48.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.24886 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-82926-1"}]': finished 2024-03-21T12:54:48.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-81661-3", "field": "max_bytes", "val": "4096"}]': finished 2024-03-21T12:54:48.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.25075 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-4", "tierpool": "test-rados-api-smithi043-81880-4-cache"}]': finished 2024-03-21T12:54:48.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.34784 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]': finished 2024-03-21T12:54:48.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.35336 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-81636-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:48.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.35342 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:48.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/524839068' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-4", "overlaypool": "test-rados-api-smithi043-81880-4-cache"}]: dispatch 2024-03-21T12:54:48.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2542191100' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:48.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1890594059' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-82926-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:48.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:48.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4129264064' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:48.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: osdmap e85: 8 total, 8 up, 8 in 2024-03-21T12:54:48.025 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.25312 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-81775-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-81775-10"}]': finished 2024-03-21T12:54:48.025 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-81755-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-81755-10"}]': finished 2024-03-21T12:54:48.025 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-82758-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-82758-2"}]': finished 2024-03-21T12:54:48.025 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: osdmap e86: 8 total, 8 up, 8 in 2024-03-21T12:54:48.025 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: pgmap v88: 804 pgs: 112 creating+peering, 548 active+clean, 144 unknown; 145 MiB data, 729 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 6 op/s 2024-03-21T12:54:48.025 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: threexx 2024-03-21T12:54:48.025 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-21T12:54:48.025 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.24886 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-82926-1"}]': finished 2024-03-21T12:54:48.025 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-81661-3", "field": "max_bytes", "val": "4096"}]': finished 2024-03-21T12:54:48.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.25075 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-4", "tierpool": "test-rados-api-smithi043-81880-4-cache"}]': finished 2024-03-21T12:54:48.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.34784 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]': finished 2024-03-21T12:54:48.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.35336 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-81636-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:48.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.35342 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:48.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/524839068' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-4", "overlaypool": "test-rados-api-smithi043-81880-4-cache"}]: dispatch 2024-03-21T12:54:48.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2542191100' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:48.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1890594059' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-82926-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:48.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:48.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4129264064' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:48.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: osdmap e85: 8 total, 8 up, 8 in 2024-03-21T12:54:48.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.25312 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-81775-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-81775-10"}]': finished 2024-03-21T12:54:48.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-81755-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-81755-10"}]': finished 2024-03-21T12:54:48.026 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-82758-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-82758-2"}]': finished 2024-03-21T12:54:48.027 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: osdmap e86: 8 total, 8 up, 8 in 2024-03-21T12:54:48.027 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.34784 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:48.027 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-4", "overlaypool": "test-rados-api-smithi043-81880-4-cache"}]: dispatch 2024-03-21T12:54:48.027 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-82926-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:48.027 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.25357 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:48.027 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:48.027 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: Health check update: 17 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:54:48.027 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.34784 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82308-1"}]': finished 2024-03-21T12:54:48.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.25075 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-4", "overlaypool": "test-rados-api-smithi043-81880-4-cache"}]': finished 2024-03-21T12:54:48.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-82926-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:48.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.25357 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:48.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: osdmap e87: 8 total, 8 up, 8 in 2024-03-21T12:54:48.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/524839068' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-4-cache", "mode": "writeback"}]: dispatch 2024-03-21T12:54:48.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1890594059' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-82926-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-82926-2"}]: dispatch 2024-03-21T12:54:48.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-4-cache", "mode": "writeback"}]: dispatch 2024-03-21T12:54:48.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1214900559' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:48.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.35270 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:48.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-82926-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-82926-2"}]: dispatch 2024-03-21T12:54:48.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3449806415' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:48.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4261994065' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:48.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.25114 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:48.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/247888446' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:48.028 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.25240 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:48.029 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[30062]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:48.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.34784 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:48.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-4", "overlaypool": "test-rados-api-smithi043-81880-4-cache"}]: dispatch 2024-03-21T12:54:48.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-82926-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:48.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.25357 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:48.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:48.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: Health check update: 17 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:54:48.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.34784 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82308-1"}]': finished 2024-03-21T12:54:48.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.25075 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-4", "overlaypool": "test-rados-api-smithi043-81880-4-cache"}]': finished 2024-03-21T12:54:48.029 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-82926-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:48.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.25357 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:48.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: osdmap e87: 8 total, 8 up, 8 in 2024-03-21T12:54:48.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/524839068' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-4-cache", "mode": "writeback"}]: dispatch 2024-03-21T12:54:48.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1890594059' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-82926-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-82926-2"}]: dispatch 2024-03-21T12:54:48.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-4-cache", "mode": "writeback"}]: dispatch 2024-03-21T12:54:48.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1214900559' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:48.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.35270 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:48.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-82926-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-82926-2"}]: dispatch 2024-03-21T12:54:48.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3449806415' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:48.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4261994065' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:48.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.25114 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:48.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/247888446' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:48.030 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.25240 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:48.031 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:47 smithi043 ceph-mon[41699]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:48.174 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: pgmap v88: 804 pgs: 112 creating+peering, 548 active+clean, 144 unknown; 145 MiB data, 729 MiB used, 715 GiB / 715 GiB avail; 12 MiB/s rd, 37 MiB/s wr, 6 op/s 2024-03-21T12:54:48.174 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: threexx 2024-03-21T12:54:48.174 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-21T12:54:48.174 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.24886 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi043-82926-1"}]': finished 2024-03-21T12:54:48.175 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.25111 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi043-81661-3", "field": "max_bytes", "val": "4096"}]': finished 2024-03-21T12:54:48.175 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.25075 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-4", "tierpool": "test-rados-api-smithi043-81880-4-cache"}]': finished 2024-03-21T12:54:48.175 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.34784 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi043-82308-1"}]': finished 2024-03-21T12:54:48.175 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.35336 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi043-81636-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:48.175 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.35342 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:48.175 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/524839068' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-4", "overlaypool": "test-rados-api-smithi043-81880-4-cache"}]: dispatch 2024-03-21T12:54:48.175 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2542191100' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:48.175 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1890594059' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-82926-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:48.175 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:48.175 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4129264064' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:48.175 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: osdmap e85: 8 total, 8 up, 8 in 2024-03-21T12:54:48.175 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.25312 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi043-81775-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi043-81775-10"}]': finished 2024-03-21T12:54:48.175 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi043-81755-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi043-81755-10"}]': finished 2024-03-21T12:54:48.175 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi043-82758-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi043-82758-2"}]': finished 2024-03-21T12:54:48.175 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: osdmap e86: 8 total, 8 up, 8 in 2024-03-21T12:54:48.176 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.34784 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82308-1"}]: dispatch 2024-03-21T12:54:48.176 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-4", "overlaypool": "test-rados-api-smithi043-81880-4-cache"}]: dispatch 2024-03-21T12:54:48.176 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-82926-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:48.176 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.25357 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:48.176 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:48.176 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: Health check update: 17 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:54:48.176 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.34784 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi043-82308-1"}]': finished 2024-03-21T12:54:48.176 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.25075 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-4", "overlaypool": "test-rados-api-smithi043-81880-4-cache"}]': finished 2024-03-21T12:54:48.176 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi043-82926-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:48.176 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.25357 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:48.176 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: osdmap e87: 8 total, 8 up, 8 in 2024-03-21T12:54:48.176 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/524839068' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-4-cache", "mode": "writeback"}]: dispatch 2024-03-21T12:54:48.176 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1890594059' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-82926-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-82926-2"}]: dispatch 2024-03-21T12:54:48.176 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-4-cache", "mode": "writeback"}]: dispatch 2024-03-21T12:54:48.177 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1214900559' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:48.177 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.35270 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:48.177 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-82926-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-82926-2"}]: dispatch 2024-03-21T12:54:48.177 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3449806415' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:48.177 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4261994065' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:48.177 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.25114 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:48.177 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/247888446' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:48.177 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.25240 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:48.177 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:47 smithi155 ceph-mon[33846]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:48.377 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:54:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:54:48.433 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:54:49.145 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-mon[33846]: pgmap v91: 956 pgs: 112 creating+peering, 548 active+clean, 296 unknown; 145 MiB data, 729 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:54:49.145 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3564148337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:49.145 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:54:49.145 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-03-21T12:54:49.145 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:49.145 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:54:49.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-mon[33846]: from='client.25075 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-4-cache", "mode": "writeback"}]': finished 2024-03-21T12:54:49.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-mon[33846]: from='client.35270 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7"}]': finished 2024-03-21T12:54:49.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-mon[33846]: from='client.25114 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81674-16"}]': finished 2024-03-21T12:54:49.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-mon[33846]: from='client.25240 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82121-7"}]': finished 2024-03-21T12:54:49.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-mon[33846]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22"}]': finished 2024-03-21T12:54:49.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3449806415' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:49.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4261994065' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:49.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1214900559' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:49.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/247888446' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:49.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/524839068' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-4"}]: dispatch 2024-03-21T12:54:49.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-mon[33846]: osdmap e88: 8 total, 8 up, 8 in 2024-03-21T12:54:49.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1869762951' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:49.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/967728174' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-81636-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:49.146 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/864729788' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:49.147 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-mon[33846]: from='client.25402 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:49.147 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3564148337' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:49.147 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:54:49.147 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:54:49.147 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:48 smithi155 ceph-mon[33846]: from='client.35270 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[30062]: pgmap v91: 956 pgs: 112 creating+peering, 548 active+clean, 296 unknown; 145 MiB data, 729 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:54:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3564148337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:54:49.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-03-21T12:54:49.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:49.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:54:49.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[30062]: from='client.25075 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-4-cache", "mode": "writeback"}]': finished 2024-03-21T12:54:49.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[30062]: from='client.35270 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7"}]': finished 2024-03-21T12:54:49.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[30062]: from='client.25114 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81674-16"}]': finished 2024-03-21T12:54:49.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[30062]: from='client.25240 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82121-7"}]': finished 2024-03-21T12:54:49.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[30062]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22"}]': finished 2024-03-21T12:54:49.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3449806415' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:49.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4261994065' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:49.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1214900559' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:49.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/247888446' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:49.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/524839068' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-4"}]: dispatch 2024-03-21T12:54:49.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[30062]: osdmap e88: 8 total, 8 up, 8 in 2024-03-21T12:54:49.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1869762951' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:49.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/967728174' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-81636-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:49.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/864729788' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:49.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[30062]: from='client.25402 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:49.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3564148337' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:49.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:54:49.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:54:49.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[30062]: from='client.35270 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:49.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[41699]: pgmap v91: 956 pgs: 112 creating+peering, 548 active+clean, 296 unknown; 145 MiB data, 729 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:54:49.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3564148337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:49.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:54:49.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-03-21T12:54:49.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:49.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:54:49.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[41699]: from='client.25075 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-4-cache", "mode": "writeback"}]': finished 2024-03-21T12:54:49.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[41699]: from='client.35270 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi043-82055-7"}]': finished 2024-03-21T12:54:49.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[41699]: from='client.25114 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi043-81674-16"}]': finished 2024-03-21T12:54:49.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[41699]: from='client.25240 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82121-7"}]': finished 2024-03-21T12:54:49.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[41699]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22"}]': finished 2024-03-21T12:54:49.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3449806415' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:49.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4261994065' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:49.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1214900559' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:49.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/247888446' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:49.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/524839068' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-4"}]: dispatch 2024-03-21T12:54:49.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[41699]: osdmap e88: 8 total, 8 up, 8 in 2024-03-21T12:54:49.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1869762951' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:49.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/967728174' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-81636-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:49.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/864729788' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:49.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[41699]: from='client.25402 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:49.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3564148337' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:49.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:54:49.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:54:49.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:48 smithi043 ceph-mon[41699]: from='client.35270 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82055-7"}]: dispatch 2024-03-21T12:54:49.321 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:47.150587+0000 mon.b [INF] from='client.? v1:172.21.15.43:0/247888446' e list: Running main() from gmock_main.cc 2024-03-21T12:54:49.321 INFO:tasks.workunit.client.0.smithi043.stdout: list: [==========] Running 3 tests from 1 test suite. 2024-03-21T12:54:49.321 INFO:tasks.workunit.client.0.smithi043.stdout: list: [----------] Global test environment set-up. 2024-03-21T12:54:49.321 INFO:tasks.workunit.client.0.smithi043.stdout: list: [----------] 3 tests from NeoradosList 2024-03-21T12:54:49.321 INFO:tasks.workunit.client.0.smithi043.stdout: list: [ RUN ] NeoradosList.ListObjects 2024-03-21T12:54:49.321 INFO:tasks.workunit.client.0.smithi043.stdout: list: [ OK ] NeoradosList.ListObjects (13905 ms) 2024-03-21T12:54:49.321 INFO:tasks.workunit.client.0.smithi043.stdout: list: [ RUN ] NeoradosList.ListObjectsNS 2024-03-21T12:54:49.321 INFO:tasks.workunit.client.0.smithi043.stdout: list: [ OK ] NeoradosList.ListObjectsNS (4296 ms) 2024-03-21T12:54:49.321 INFO:tasks.workunit.client.0.smithi043.stdout: list: [ RUN ] NeoradosList.ListObjectsMany 2024-03-21T12:54:49.321 INFO:tasks.workunit.client.0.smithi043.stdout: list: [ OK ] NeoradosList.ListObjectsMany (5102 ms) 2024-03-21T12:54:49.321 INFO:tasks.workunit.client.0.smithi043.stdout: list: [----------] 3 tests from NeoradosList (23303 ms total) 2024-03-21T12:54:49.322 INFO:tasks.workunit.client.0.smithi043.stdout: list: 2024-03-21T12:54:49.322 INFO:tasks.workunit.client.0.smithi043.stdout: list: [----------] Global test environment tear-down 2024-03-21T12:54:49.322 INFO:tasks.workunit.client.0.smithi043.stdout: list: [==========] 3 tests from 1 test suite ran. (23303 ms total) 2024-03-21T12:54:49.322 INFO:tasks.workunit.client.0.smithi043.stdout: list: [ PASSED ] 3 tests. 2024-03-21T12:54:49.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-03-21T12:54:49.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-03-21T12:54:49.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-03-21T12:54:49.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-03-21T12:54:49.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: seed 82055 2024-03-21T12:54:49.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-03-21T12:54:49.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (380 ms) 2024-03-21T12:54:49.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-03-21T12:54:49.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (6 ms) 2024-03-21T12:54:49.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-03-21T12:54:49.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (0 ms) 2024-03-21T12:54:49.325 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-03-21T12:54:49.326 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (9 ms) 2024-03-21T12:54:49.326 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-03-21T12:54:49.326 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (18 ms) 2024-03-21T12:54:49.326 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (413 ms total) 2024-03-21T12:54:49.326 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: 2024-03-21T12:54:49.326 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-03-21T12:54:49.326 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-03-21T12:54:49.326 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (1193 ms) 2024-03-21T12:54:49.326 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-03-21T12:54:49.326 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (1 ms) 2024-03-21T12:54:49.326 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-03-21T12:54:49.326 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (3 ms) 2024-03-21T12:54:49.326 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-03-21T12:54:49.326 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (10 ms) 2024-03-21T12:54:49.326 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (1207 ms total) 2024-03-21T12:54:49.327 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: 2024-03-21T12:54:49.327 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-03-21T12:54:49.327 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (23584 ms total) 2024-03-21T12:54:49.327 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-03-21T12:54:49.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: Running main() from gmock_main.cc 2024-03-21T12:54:49.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-03-21T12:54:49.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [----------] Global test environment set-up. 2024-03-21T12:54:49.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-03-21T12:54:49.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-03-21T12:54:49.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (85 ms) 2024-03-21T12:54:49.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-03-21T12:54:49.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-03-21T12:54:49.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-03-21T12:54:49.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: no timeout :/ 2024-03-21T12:54:49.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: no timeout :/ 2024-03-21T12:54:49.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: no timeout :/ 2024-03-21T12:54:49.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: no timeout :/ 2024-03-21T12:54:49.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: no timeout :/ 2024-03-21T12:54:49.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (28 ms) 2024-03-21T12:54:49.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-03-21T12:54:49.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (3 ms) 2024-03-21T12:54:49.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-03-21T12:54:49.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.Checksum (3 ms) 2024-03-21T12:54:49.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-03-21T12:54:49.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (5 ms) 2024-03-21T12:54:49.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-03-21T12:54:49.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (3 ms) 2024-03-21T12:54:49.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-03-21T12:54:49.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (2 ms) 2024-03-21T12:54:49.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-03-21T12:54:49.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (1 ms) 2024-03-21T12:54:49.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-03-21T12:54:49.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.TruncTest (3 ms) 2024-03-21T12:54:49.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-03-21T12:54:49.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (3 ms) 2024-03-21T12:54:49.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-03-21T12:54:49.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (2 ms) 2024-03-21T12:54:49.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-03-21T12:54:49.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.RmXattr (8 ms) 2024-03-21T12:54:49.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-03-21T12:54:49.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIo.XattrIter (3 ms) 2024-03-21T12:54:49.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [----------] 14 tests from LibRadosIo (150 ms total) 2024-03-21T12:54:49.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: 2024-03-21T12:54:49.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-03-21T12:54:49.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-03-21T12:54:49.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (1167 ms) 2024-03-21T12:54:49.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-03-21T12:54:49.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (13 ms) 2024-03-21T12:54:49.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-03-21T12:54:49.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (11 ms) 2024-03-21T12:54:49.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-03-21T12:54:49.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (6 ms) 2024-03-21T12:54:49.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-03-21T12:54:49.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (7 ms) 2024-03-21T12:54:49.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-03-21T12:54:49.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (3 ms) 2024-03-21T12:54:49.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-03-21T12:54:49.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (3 ms) 2024-03-21T12:54:49.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-03-21T12:54:49.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (3 ms) 2024-03-21T12:54:49.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-03-21T12:54:49.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (9 ms) 2024-03-21T12:54:49.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-03-21T12:54:49.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (4 ms) 2024-03-21T12:54:49.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [----------] 10 tests from LibRadosIoEC (1227 ms total) 2024-03-21T12:54:49.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: 2024-03-21T12:54:49.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [-------ibRadosIoECPP.OverlappingWriteRoundTripPP 2024-03-21T12:54:49.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (3 ms) 2024-03-21T12:54:49.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-03-21T12:54:49.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (6 ms) 2024-03-21T12:54:49.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-03-21T12:54:49.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (2 ms) 2024-03-21T12:54:49.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-03-21T12:54:49.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (6 ms) 2024-03-21T12:54:49.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-03-21T12:54:49.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (3 ms) 2024-03-21T12:54:49.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-03-21T12:54:49.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (3 ms) 2024-03-21T12:54:49.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-03-21T12:54:49.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (2 ms) 2024-03-21T12:54:49.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-03-21T12:54:49.346 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (8 ms) 2024-03-21T12:54:49.347 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-03-21T12:54:49.347 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (4 ms) 2024-03-21T12:54:49.347 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-03-21T12:54:49.347 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (4 ms) 2024-03-21T12:54:49.347 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-03-21T12:54:49.347 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (2 ms) 2024-03-21T12:54:49.347 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-03-21T12:54:49.347 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (4 ms) 2024-03-21T12:54:49.347 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (1271 ms total) 2024-03-21T12:54:49.347 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: 2024-03-21T12:54:49.347 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [----------] Global test environment tear-down 2024-03-21T12:54:49.347 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (23721 ms total) 2024-03-21T12:54:49.347 INFO:tasks.workunit.client.0.smithi043.stdout: api_io_pp: [ PASSED ] 37 tests. 2024-03-21T12:54:49.347 INFO:tasks.workunit.client.0.smithi043.stdout:---] Global test environment tear-down 2024-03-21T12:54:49.347 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [==========] 24 tests from 2 test suites ran. (23725 ms total) 2024-03-21T12:54:49.347 INFO:tasks.workunit.client.0.smithi043.stdout: api_io: [ PASSED ] 24 tests. 2024-03-21T12:54:49.349 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: Running main() from gmock_main.cc 2024-03-21T12:54:49.350 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [==========] Running 9 tests from 2 test suites. 2024-03-21T12:54:49.350 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [----------] Global test environment set-up. 2024-03-21T12:54:49.350 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [----------] 5 tests from LibRadosStat 2024-03-21T12:54:49.350 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-03-21T12:54:49.350 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStat.Stat (396 ms) 2024-03-21T12:54:49.350 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStat.Stat2 2024-03-21T12:54:49.350 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStat.Stat2 (4 ms) 2024-03-21T12:54:49.350 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-03-21T12:54:49.350 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStat.StatNS (14 ms) 2024-03-21T12:54:49.350 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-03-21T12:54:49.350 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (0 ms) 2024-03-21T12:54:49.350 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-03-21T12:54:49.350 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (3 ms) 2024-03-21T12:54:49.350 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [----------] 5 tests from LibRadosStat (417 ms total) 2024-03-21T12:54:49.351 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: 2024-03-21T12:54:49.351 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-03-21T12:54:49.351 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-03-21T12:54:49.351 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (1148 ms) 2024-03-21T12:54:49.351 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-03-21T12:54:49.351 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (51 ms) 2024-03-21T12:54:49.351 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-03-21T12:54:49.351 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (1 ms) 2024-03-21T12:54:49.351 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-03-21T12:54:49.351 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (2 ms) 2024-03-21T12:54:49.351 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (1202 ms total) 2024-03-21T12:54:49.351 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: 2024-03-21T12:54:49.351 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [----------] Global test environment tear-down 2024-03-21T12:54:49.351 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [==========] 9 tests from 2 test suites ran. (23580 ms total) 2024-03-21T12:54:49.351 INFO:tasks.workunit.client.0.smithi043.stdout: api_stat: [ PASSED ] 9 tests. 2024-03-21T12:54:50.451 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:54:50.451 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: fourxx 2024-03-21T12:54:50.451 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:50.451 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-03-21T12:54:50.451 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.25114 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:50.451 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.25240 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:50.451 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:50.451 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.35396 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:50.451 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.35399 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:50.451 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-4"}]: dispatch 2024-03-21T12:54:50.452 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.25387 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-81636-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:50.452 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.25402 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:50.452 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3564148337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82129-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:50.452 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.25402 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82129-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:50.452 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-82926-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-82926-2"}]': finished 2024-03-21T12:54:50.452 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.35270 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82055-7"}]': finished 2024-03-21T12:54:50.452 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.25114 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81674-16"}]': finished 2024-03-21T12:54:50.452 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.25240 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82121-7"}]': finished 2024-03-21T12:54:50.452 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81695-22"}]': finished 2024-03-21T12:54:50.452 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.35396 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:50.452 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:54:50.453 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: fourxx 2024-03-21T12:54:50.453 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:50.453 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-03-21T12:54:50.453 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.25114 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:50.453 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.25240 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:50.453 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:50.453 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.35396 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:50.453 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.35399 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:50.453 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-4"}]: dispatch 2024-03-21T12:54:50.453 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.25387 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-81636-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:50.453 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.25402 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:50.453 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3564148337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82129-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:50.453 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.25402 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82129-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:50.453 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-82926-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-82926-2"}]': finished 2024-03-21T12:54:50.453 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.35270 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82055-7"}]': finished 2024-03-21T12:54:50.453 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.25114 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81674-16"}]': finished 2024-03-21T12:54:50.454 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.25240 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82121-7"}]': finished 2024-03-21T12:54:50.454 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81695-22"}]': finished 2024-03-21T12:54:50.454 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.35396 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:50.454 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.35399 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:50.454 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.25075 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-4"}]': finished 2024-03-21T12:54:50.454 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.25387 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-81636-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:50.454 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.25402 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82129-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:50.454 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3564148337' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-82129-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:50.454 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/524839068' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-4", "tierpool": "test-rados-api-smithi043-81880-4-cache"}]: dispatch 2024-03-21T12:54:50.454 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: osdmap e89: 8 total, 8 up, 8 in 2024-03-21T12:54:50.455 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4010074928' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-82758-2"}]: dispatch 2024-03-21T12:54:50.455 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-82758-2"}]: dispatch 2024-03-21T12:54:50.455 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.25402 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-82129-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:50.455 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-4", "tierpool": "test-rados-api-smithi043-81880-4-cache"}]: dispatch 2024-03-21T12:54:50.455 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1869762951' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-82308-6", "pg_num": 4}]: dispatch 2024-03-21T12:54:50.455 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.35399 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:50.455 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.25075 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-4"}]': finished 2024-03-21T12:54:50.455 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.25387 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-81636-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:50.456 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.25402 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82129-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:50.456 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3564148337' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-82129-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:50.456 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/524839068' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-4", "tierpool": "test-rados-api-smithi043-81880-4-cache"}]: dispatch 2024-03-21T12:54:50.456 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: osdmap e89: 8 total, 8 up, 8 in 2024-03-21T12:54:50.456 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4010074928' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-82758-2"}]: dispatch 2024-03-21T12:54:50.456 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-82758-2"}]: dispatch 2024-03-21T12:54:50.456 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.25402 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-82129-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:50.456 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-4", "tierpool": "test-rados-api-smithi043-81880-4-cache"}]: dispatch 2024-03-21T12:54:50.456 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:50 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1869762951' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-82308-6", "pg_num": 4}]: dispatch 2024-03-21T12:54:50.461 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:54:50.461 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: fourxx 2024-03-21T12:54:50.461 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:50.461 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-03-21T12:54:50.461 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.25114 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81674-16"}]: dispatch 2024-03-21T12:54:50.461 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.25240 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:50.461 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:50.461 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.35396 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:50.461 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.35399 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:50.461 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-4"}]: dispatch 2024-03-21T12:54:50.462 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.25387 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-81636-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:50.462 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.25402 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:50.462 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3564148337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82129-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:50.462 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.25402 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82129-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:50.462 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi043-82926-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi043-82926-2"}]': finished 2024-03-21T12:54:50.462 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.35270 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi043-82055-7"}]': finished 2024-03-21T12:54:50.462 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.25114 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi043-81674-16"}]': finished 2024-03-21T12:54:50.462 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.25240 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi043-82121-7"}]': finished 2024-03-21T12:54:50.462 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.25120 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi043-81695-22"}]': finished 2024-03-21T12:54:50.462 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.35396 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:50.462 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.35399 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:50.462 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.25075 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-4"}]': finished 2024-03-21T12:54:50.462 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.25387 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi043-81636-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:50.463 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.25402 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82129-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:50.463 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3564148337' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-82129-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:50.463 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/524839068' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-4", "tierpool": "test-rados-api-smithi043-81880-4-cache"}]: dispatch 2024-03-21T12:54:50.463 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: osdmap e89: 8 total, 8 up, 8 in 2024-03-21T12:54:50.463 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4010074928' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-82758-2"}]: dispatch 2024-03-21T12:54:50.463 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-82758-2"}]: dispatch 2024-03-21T12:54:50.463 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.25402 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-82129-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:50.463 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.25075 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-4", "tierpool": "test-rados-api-smithi043-81880-4-cache"}]: dispatch 2024-03-21T12:54:50.463 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:50 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1869762951' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-82308-6", "pg_num": 4}]: dispatch 2024-03-21T12:54:51.454 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: Running main() from gmock_main.cc 2024-03-21T12:54:51.454 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [==========] Running 2 tests from 1 test suite. 2024-03-21T12:54:51.455 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [----------] Global test environment set-up. 2024-03-21T12:54:51.455 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [----------] 2 tests from NeoRadosECIo 2024-03-21T12:54:51.455 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [ RUN ] NeoRadosECIo.SimpleWrite 2024-03-21T12:54:51.455 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [ OK ] NeoRadosECIo.SimpleWrite (16155 ms) 2024-03-21T12:54:51.455 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [ RUN ] NeoRadosECIo.ReadOp 2024-03-21T12:54:51.455 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [ OK ] NeoRadosECIo.ReadOp (9309 ms) 2024-03-21T12:54:51.455 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [----------] 2 tests from NeoRadosECIo (25464 ms total) 2024-03-21T12:54:51.455 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: 2024-03-21T12:54:51.455 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [----------] Global test environment tear-down 2024-03-21T12:54:51.455 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [==========] 2 tests from 1 test suite ran. (25464 ms total) 2024-03-21T12:54:51.455 INFO:tasks.workunit.client.0.smithi043.stdout: ec_io: [ PASSED ] 2 tests. 2024-03-21T12:54:51.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[41699]: pgmap v94: 860 pgs: 732 active+clean, 128 unknown; 145 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 4.0 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-03-21T12:54:51.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:51.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-03-21T12:54:51.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[41699]: from='client.35396 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-82308-6", "pg_num": 4}]: dispatch 2024-03-21T12:54:51.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:54:51.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[41699]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-82758-2"}]': finished 2024-03-21T12:54:51.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[41699]: from='client.25075 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-4", "tierpool": "test-rados-api-smithi043-81880-4-cache"}]': finished 2024-03-21T12:54:51.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[41699]: from='client.35396 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-82308-6", "pg_num": 4}]': finished 2024-03-21T12:54:51.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4010074928' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-82758-2"}]: dispatch 2024-03-21T12:54:51.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1869762951' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "tierpool": "test-rados-api-smithi043-82308-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:54:51.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1071408181' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:51.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[41699]: osdmap e90: 8 total, 8 up, 8 in 2024-03-21T12:54:51.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[41699]: from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-82758-2"}]: dispatch 2024-03-21T12:54:51.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[41699]: from='client.35396 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "tierpool": "test-rados-api-smithi043-82308-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:54:51.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:51.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[41699]: from='client.25423 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:51.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[30062]: pgmap v94: 860 pgs: 732 active+clean, 128 unknown; 145 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 4.0 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-03-21T12:54:51.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:51.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-03-21T12:54:51.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[30062]: from='client.35396 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-82308-6", "pg_num": 4}]: dispatch 2024-03-21T12:54:51.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:54:51.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[30062]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-82758-2"}]': finished 2024-03-21T12:54:51.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[30062]: from='client.25075 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-4", "tierpool": "test-rados-api-smithi043-81880-4-cache"}]': finished 2024-03-21T12:54:51.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[30062]: from='client.35396 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-82308-6", "pg_num": 4}]': finished 2024-03-21T12:54:51.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4010074928' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-82758-2"}]: dispatch 2024-03-21T12:54:51.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1869762951' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "tierpool": "test-rados-api-smithi043-82308-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:54:51.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1071408181' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:51.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[30062]: osdmap e90: 8 total, 8 up, 8 in 2024-03-21T12:54:51.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[30062]: from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-82758-2"}]: dispatch 2024-03-21T12:54:51.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[30062]: from='client.35396 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "tierpool": "test-rados-api-smithi043-82308-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:54:51.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:51.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:51 smithi043 ceph-mon[30062]: from='client.25423 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:51.618 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:51 smithi155 ceph-mon[33846]: pgmap v94: 860 pgs: 732 active+clean, 128 unknown; 145 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 4.0 KiB/s rd, 2.5 KiB/s wr, 7 op/s 2024-03-21T12:54:51.618 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:51 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:51.618 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:51 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3417849448' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-03-21T12:54:51.618 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:51 smithi155 ceph-mon[33846]: from='client.35396 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-82308-6", "pg_num": 4}]: dispatch 2024-03-21T12:54:51.618 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:51 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:54:51.618 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:51 smithi155 ceph-mon[33846]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi043-82758-2"}]': finished 2024-03-21T12:54:51.619 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:51 smithi155 ceph-mon[33846]: from='client.25075 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-4", "tierpool": "test-rados-api-smithi043-81880-4-cache"}]': finished 2024-03-21T12:54:51.619 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:51 smithi155 ceph-mon[33846]: from='client.35396 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-82308-6", "pg_num": 4}]': finished 2024-03-21T12:54:51.619 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:51 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4010074928' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-82758-2"}]: dispatch 2024-03-21T12:54:51.619 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:51 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1869762951' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "tierpool": "test-rados-api-smithi043-82308-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:54:51.619 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:51 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1071408181' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:51.619 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:51 smithi155 ceph-mon[33846]: osdmap e90: 8 total, 8 up, 8 in 2024-03-21T12:54:51.619 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:51 smithi155 ceph-mon[33846]: from='client.35330 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-82758-2"}]: dispatch 2024-03-21T12:54:51.619 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:51 smithi155 ceph-mon[33846]: from='client.35396 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "tierpool": "test-rados-api-smithi043-82308-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:54:51.619 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:51 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:51.619 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:51 smithi155 ceph-mon[33846]: from='client.25423 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:52.172 INFO:tasks.workunit.client.0.smithi043.stdout:ntity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:52.172 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:47.151000+0000 mon.a [INF] from='client.25240 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi043-82121-7"}]: dispatch 2024-03-21T12:54:52.172 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: got: 2024-03-21T12:54:47.151335+0000 mon.a [INF] from='client.25120 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi043-81695-22"}]: dispatch 2024-03-21T12:54:52.173 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (12249 ms) 2024-03-21T12:54:52.173 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (26387 ms total) 2024-03-21T12:54:52.173 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: 2024-03-21T12:54:52.173 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [----------] Global test environment tear-down 2024-03-21T12:54:52.173 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (26387 ms total) 2024-03-21T12:54:52.173 INFO:tasks.workunit.client.0.smithi043.stdout: api_cmd: [ PASSED ] 4 tests. 2024-03-21T12:54:53.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: pgmap v97: 832 pgs: 596 active+clean, 236 unknown; 145 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T12:54:53.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.25402 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-82129-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-82129-12"}]': finished 2024-03-21T12:54:53.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-82758-2"}]': finished 2024-03-21T12:54:53.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.35396 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "tierpool": "test-rados-api-smithi043-82308-6", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:54:53.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.25423 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:53.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: osdmap e91: 8 total, 8 up, 8 in 2024-03-21T12:54:53.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1869762951' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "overlaypool": "test-rados-api-smithi043-82308-6"}]: dispatch 2024-03-21T12:54:53.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/282144841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:53.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1213584964' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:53.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3458186974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:53.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1890594059' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-82926-2"}]: dispatch 2024-03-21T12:54:53.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.35396 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "overlaypool": "test-rados-api-smithi043-82308-6"}]: dispatch 2024-03-21T12:54:53.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.35438 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:53.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.25312 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:53.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:53.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-82926-2"}]: dispatch 2024-03-21T12:54:53.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:53.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: Health check update: 14 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:54:53.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: pool 'PoolQuotaPP_smithi043-81661-3' is full (reached quota's max_bytes: 4 KiB) 2024-03-21T12:54:53.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-21T12:54:53.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.35396 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "overlaypool": "test-rados-api-smithi043-82308-6"}]': finished 2024-03-21T12:54:53.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.35438 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:53.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.25312 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81775-10"}]': finished 2024-03-21T12:54:53.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10"}]': finished 2024-03-21T12:54:53.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-82926-2"}]': finished 2024-03-21T12:54:53.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/282144841' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:53.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: osdmap e92: 8 total, 8 up, 8 in 2024-03-21T12:54:53.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1890594059' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-82926-2"}]: dispatch 2024-03-21T12:54:53.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1869762951' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-82308-6", "mode": "writeback"}]: dispatch 2024-03-21T12:54:53.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3458186974' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:53.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1278130657' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-81636-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:53.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.35396 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-82308-6", "mode": "writeback"}]: dispatch 2024-03-21T12:54:53.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:53.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.35453 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-81636-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:53.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:54:53.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.25312 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:53.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-82926-2"}]: dispatch 2024-03-21T12:54:53.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:54:53.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:52 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:54:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: pgmap v97: 832 pgs: 596 active+clean, 236 unknown; 145 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T12:54:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.25402 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-82129-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-82129-12"}]': finished 2024-03-21T12:54:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-82758-2"}]': finished 2024-03-21T12:54:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.35396 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "tierpool": "test-rados-api-smithi043-82308-6", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:54:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.25423 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: osdmap e91: 8 total, 8 up, 8 in 2024-03-21T12:54:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1869762951' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "overlaypool": "test-rados-api-smithi043-82308-6"}]: dispatch 2024-03-21T12:54:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/282144841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1213584964' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:53.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3458186974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:53.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1890594059' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-82926-2"}]: dispatch 2024-03-21T12:54:53.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.35396 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "overlaypool": "test-rados-api-smithi043-82308-6"}]: dispatch 2024-03-21T12:54:53.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.35438 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:53.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.25312 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:53.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:53.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-82926-2"}]: dispatch 2024-03-21T12:54:53.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:53.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: Health check update: 14 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:54:53.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: pool 'PoolQuotaPP_smithi043-81661-3' is full (reached quota's max_bytes: 4 KiB) 2024-03-21T12:54:53.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-21T12:54:53.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: pgmap v97: 832 pgs: 596 active+clean, 236 unknown; 145 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T12:54:53.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.25402 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi043-82129-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi043-82129-12"}]': finished 2024-03-21T12:54:53.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.35330 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi043-82758-2"}]': finished 2024-03-21T12:54:53.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.35396 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "tierpool": "test-rados-api-smithi043-82308-6", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:54:53.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.25423 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:53.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: osdmap e91: 8 total, 8 up, 8 in 2024-03-21T12:54:53.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1869762951' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "overlaypool": "test-rados-api-smithi043-82308-6"}]: dispatch 2024-03-21T12:54:53.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/282144841' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:53.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1213584964' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:53.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3458186974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:53.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1890594059' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-82926-2"}]: dispatch 2024-03-21T12:54:53.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.35396 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "overlaypool": "test-rados-api-smithi043-82308-6"}]: dispatch 2024-03-21T12:54:53.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.35438 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:53.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.25312 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:53.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:53.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-82926-2"}]: dispatch 2024-03-21T12:54:53.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:53.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: Health check update: 14 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:54:53.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: pool 'PoolQuotaPP_smithi043-81661-3' is full (reached quota's max_bytes: 4 KiB) 2024-03-21T12:54:53.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-21T12:54:53.237 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.35396 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "overlaypool": "test-rados-api-smithi043-82308-6"}]': finished 2024-03-21T12:54:53.237 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.35438 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:53.237 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.25312 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81775-10"}]': finished 2024-03-21T12:54:53.237 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10"}]': finished 2024-03-21T12:54:53.237 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-82926-2"}]': finished 2024-03-21T12:54:53.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/282144841' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:53.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: osdmap e92: 8 total, 8 up, 8 in 2024-03-21T12:54:53.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1890594059' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-82926-2"}]: dispatch 2024-03-21T12:54:53.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1869762951' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-82308-6", "mode": "writeback"}]: dispatch 2024-03-21T12:54:53.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3458186974' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:53.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1278130657' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-81636-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:53.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.35396 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-82308-6", "mode": "writeback"}]: dispatch 2024-03-21T12:54:53.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:53.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.35453 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-81636-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:53.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:54:53.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.25312 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:53.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-82926-2"}]: dispatch 2024-03-21T12:54:53.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:54:53.238 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:54:53.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.35396 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "overlaypool": "test-rados-api-smithi043-82308-6"}]': finished 2024-03-21T12:54:53.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.35438 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:53.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.25312 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi043-81775-10"}]': finished 2024-03-21T12:54:53.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi043-81755-10"}]': finished 2024-03-21T12:54:53.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi043-82926-2"}]': finished 2024-03-21T12:54:53.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/282144841' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:53.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: osdmap e92: 8 total, 8 up, 8 in 2024-03-21T12:54:53.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1890594059' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-82926-2"}]: dispatch 2024-03-21T12:54:53.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1869762951' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-82308-6", "mode": "writeback"}]: dispatch 2024-03-21T12:54:53.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3458186974' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:53.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1278130657' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-81636-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:53.239 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.35396 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-82308-6", "mode": "writeback"}]: dispatch 2024-03-21T12:54:53.240 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.35318 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81755-10"}]: dispatch 2024-03-21T12:54:53.240 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.35453 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-81636-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:53.240 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:54:53.240 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.25312 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81775-10"}]: dispatch 2024-03-21T12:54:53.240 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='client.25360 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-82926-2"}]: dispatch 2024-03-21T12:54:53.240 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:54:53.240 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:52 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:54:53.302 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-03-21T12:54:53.303 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-03-21T12:54:53.303 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-03-21T12:54:53.303 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-03-21T12:54:53.303 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: seed 81755 2024-03-21T12:54:53.303 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-03-21T12:54:53.303 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (381 ms) 2024-03-21T12:54:53.303 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-03-21T12:54:53.303 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (3 ms) 2024-03-21T12:54:53.303 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-03-21T12:54:53.303 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1006 ms) 2024-03-21T12:54:53.303 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-03-21T12:54:53.303 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1011 ms) 2024-03-21T12:54:53.303 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-03-21T12:54:53.303 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (12 ms) 2024-03-21T12:54:53.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-03-21T12:54:53.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (5 ms) 2024-03-21T12:54:53.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-03-21T12:54:53.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (12 ms) 2024-03-21T12:54:53.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-03-21T12:54:53.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (6 ms) 2024-03-21T12:54:53.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2436 ms total) 2024-03-21T12:54:53.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: 2024-03-21T12:54:53.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-03-21T12:54:53.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-03-21T12:54:53.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (966 ms) 2024-03-21T12:54:53.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-03-21T12:54:53.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (6 ms) 2024-03-21T12:54:53.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-03-21T12:54:53.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1029 ms) 2024-03-21T12:54:53.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-03-21T12:54:53.305 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1070 ms) 2024-03-21T12:54:53.305 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-03-21T12:54:53.305 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (25 ms) 2024-03-21T12:54:53.305 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-03-21T12:54:53.305 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (6 ms) 2024-03-21T12:54:53.305 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-03-21T12:54:53.305 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (5 ms) 2024-03-21T12:54:53.305 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-03-21T12:54:53.305 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (3 ms) 2024-03-21T12:54:53.305 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (3110 ms total) 2024-03-21T12:54:53.305 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: 2024-03-21T12:54:53.305 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-03-21T12:54:53.305 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (27653 ms total) 2024-03-21T12:54:53.305 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-03-21T12:54:53.310 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: Running main() from gmock_main.cc 2024-03-21T12:54:53.310 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-03-21T12:54:53.310 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [----------] Global test environment set-up. 2024-03-21T12:54:53.310 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-03-21T12:54:53.310 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-03-21T12:54:53.310 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (372 ms) 2024-03-21T12:54:53.310 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-03-21T12:54:53.310 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.LockShared (5 ms) 2024-03-21T12:54:53.311 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-03-21T12:54:53.311 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1006 ms) 2024-03-21T12:54:53.311 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-03-21T12:54:53.311 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1014 ms) 2024-03-21T12:54:53.311 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-03-21T12:54:53.311 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (14 ms) 2024-03-21T12:54:53.311 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-03-21T12:54:53.311 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.Unlock (4 ms) 2024-03-21T12:54:53.311 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-03-21T12:54:53.311 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (5 ms) 2024-03-21T12:54:53.311 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-03-21T12:54:53.311 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (8 ms) 2024-03-21T12:54:53.311 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [----------] 8 tests from LibRadosLock (2428 ms total) 2024-03-21T12:54:53.311 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: 2024-03-21T12:54:53.311 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-03-21T12:54:53.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-03-21T12:54:53.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (948 ms) 2024-03-21T12:54:53.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-03-21T12:54:53.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (14 ms) 2024-03-21T12:54:53.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-03-21T12:54:53.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1032 ms) 2024-03-21T12:54:53.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-03-21T12:54:53.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1072 ms) 2024-03-21T12:54:53.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-03-21T12:54:53.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (23 ms) 2024-03-21T12:54:53.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-03-21T12:54:53.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (7 ms) 2024-03-21T12:54:53.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-03-21T12:54:53.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (6 ms) 2024-03-21T12:54:53.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-03-21T12:54:53.312 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (4 ms) 2024-03-21T12:54:53.313 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (3106 ms total) 2024-03-21T12:54:53.313 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: 2024-03-21T12:54:53.313 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [----------] Global test environment tear-down 2024-03-21T12:54:53.313 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (27651 ms total) 2024-03-21T12:54:53.313 INFO:tasks.workunit.client.0.smithi043.stdout: api_lock: [ PASSED ] 16 tests. 2024-03-21T12:54:54.400 INFO:tasks.workunit.client.0.smithi043.stdout: pool: Running main() from gmock_main.cc 2024-03-21T12:54:54.400 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [==========] Running 6 tests from 1 test suite. 2024-03-21T12:54:54.400 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [----------] Global test environment set-up. 2024-03-21T12:54:54.400 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [----------] 6 tests from NeoRadosPools 2024-03-21T12:54:54.400 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ RUN ] NeoRadosPools.PoolList 2024-03-21T12:54:54.400 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ OK ] NeoRadosPools.PoolList (10181 ms) 2024-03-21T12:54:54.400 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ RUN ] NeoRadosPools.PoolLookup 2024-03-21T12:54:54.400 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ OK ] NeoRadosPools.PoolLookup (4844 ms) 2024-03-21T12:54:54.401 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ RUN ] NeoRadosPools.PoolLookupOtherInstance 2024-03-21T12:54:54.401 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ OK ] NeoRadosPools.PoolLookupOtherInstance (5030 ms) 2024-03-21T12:54:54.401 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ RUN ] NeoRadosPools.PoolDelete 2024-03-21T12:54:54.401 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ OK ] NeoRadosPools.PoolDelete (4384 ms) 2024-03-21T12:54:54.401 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateDelete 2024-03-21T12:54:54.401 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ OK ] NeoRadosPools.PoolCreateDelete (1829 ms) 2024-03-21T12:54:54.401 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateWithCrushRule 2024-03-21T12:54:54.401 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ OK ] NeoRadosPools.PoolCreateWithCrushRule (2099 ms) 2024-03-21T12:54:54.401 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [----------] 6 tests from NeoRadosPools (28367 ms total) 2024-03-21T12:54:54.401 INFO:tasks.workunit.client.0.smithi043.stdout: pool: 2024-03-21T12:54:54.401 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [----------] Global test environment tear-down 2024-03-21T12:54:54.401 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [==========] 6 tests from 1 test suite ran. (28367 ms total) 2024-03-21T12:54:54.401 INFO:tasks.workunit.client.0.smithi043.stdout: pool: [ PASSED ] 6 tests. 2024-03-21T12:54:54.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:54 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:54:54.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:54 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:54.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:54 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:54:54.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:54 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:54:54.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:54 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:54:54.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:54 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:54:54.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:54 smithi155 ceph-mon[33846]: from='client.35396 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-82308-6", "mode": "writeback"}]': finished 2024-03-21T12:54:54.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:54 smithi155 ceph-mon[33846]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81755-10"}]': finished 2024-03-21T12:54:54.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:54 smithi155 ceph-mon[33846]: from='client.35453 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-81636-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:54.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:54 smithi155 ceph-mon[33846]: from='client.25312 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81775-10"}]': finished 2024-03-21T12:54:54.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:54 smithi155 ceph-mon[33846]: from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-82926-2"}]': finished 2024-03-21T12:54:54.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:54 smithi155 ceph-mon[33846]: osdmap e93: 8 total, 8 up, 8 in 2024-03-21T12:54:54.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:54 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3564148337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:54.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:54 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3737344254' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-81984-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:54.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:54 smithi155 ceph-mon[33846]: from='client.25402 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:54.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:54 smithi155 ceph-mon[33846]: from='client.25468 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-81984-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:54.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:54 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/468648503' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi043-82926-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:54:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:54:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:54:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:54:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:54:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[30062]: from='client.35396 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-82308-6", "mode": "writeback"}]': finished 2024-03-21T12:54:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[30062]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81755-10"}]': finished 2024-03-21T12:54:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[30062]: from='client.35453 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-81636-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:54.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[30062]: from='client.25312 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81775-10"}]': finished 2024-03-21T12:54:54.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[30062]: from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-82926-2"}]': finished 2024-03-21T12:54:54.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[30062]: osdmap e93: 8 total, 8 up, 8 in 2024-03-21T12:54:54.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3564148337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:54.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3737344254' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-81984-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:54.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[30062]: from='client.25402 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:54.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[30062]: from='client.25468 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-81984-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:54.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/468648503' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi043-82926-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:54.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:54:54.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:54.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:54:54.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:54:54.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:54:54.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:54:54.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[41699]: from='client.35396 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-82308-6", "mode": "writeback"}]': finished 2024-03-21T12:54:54.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[41699]: from='client.35318 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi043-81755-10"}]': finished 2024-03-21T12:54:54.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[41699]: from='client.35453 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi043-81636-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:54.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[41699]: from='client.25312 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi043-81775-10"}]': finished 2024-03-21T12:54:54.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[41699]: from='client.25360 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi043-82926-2"}]': finished 2024-03-21T12:54:54.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[41699]: osdmap e93: 8 total, 8 up, 8 in 2024-03-21T12:54:54.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3564148337' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:54.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3737344254' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-81984-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:54.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[41699]: from='client.25402 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:54.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[41699]: from='client.25468 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-81984-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:54.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:54 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/468648503' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi043-82926-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:55.421 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: Running main() from gmock_main.cc 2024-03-21T12:54:55.421 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-03-21T12:54:55.421 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-03-21T12:54:55.421 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-03-21T12:54:55.421 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-03-21T12:54:55.421 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify_test_cb 2024-03-21T12:54:55.421 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (397 ms) 2024-03-21T12:54:55.421 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-03-21T12:54:55.421 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94780047454064 err -107 2024-03-21T12:54:55.421 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-03-21T12:54:55.421 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (14 ms) 2024-03-21T12:54:55.421 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-03-21T12:54:55.421 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94780047456192 err -107 2024-03-21T12:54:55.422 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-03-21T12:54:55.422 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (12 ms) 2024-03-21T12:54:55.422 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-03-21T12:54:55.422 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24757 notify_id 330712481792 cookie 94780047478576 2024-03-21T12:54:55.422 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (7 ms) 2024-03-21T12:54:55.422 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-03-21T12:54:55.422 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24757 notify_id 330712481792 cookie 94780047485760 2024-03-21T12:54:55.422 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (10 ms) 2024-03-21T12:54:55.422 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-03-21T12:54:55.422 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24757 notify_id 330712481793 cookie 94780047487328 2024-03-21T12:54:55.422 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (9 ms) 2024-03-21T12:54:55.422 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-03-21T12:54:55.422 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24757 notify_id 330712481792 cookie 94780047489648 2024-03-21T12:54:55.422 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24757 notify_id 330712481792 cookie 94780047528912 2024-03-21T12:54:55.423 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (10 ms) 2024-03-21T12:54:55.423 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-03-21T12:54:55.423 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24757 notify_id 330712481793 cookie 94780047489648 2024-03-21T12:54:55.423 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24757 notify_id 335007449090 cookie 94780047489648 2024-03-21T12:54:55.423 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3007 ms) 2024-03-21T12:54:55.423 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-03-21T12:54:55.423 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-03-21T12:54:55.423 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94780047489648 err -107 2024-03-21T12:54:55.423 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_cb from 24757 notify_id 360777252866 cookie 94780047489648 2024-03-21T12:54:55.423 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5041 ms) 2024-03-21T12:54:55.423 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-03-21T12:54:55.423 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94780047489648 err -107 2024-03-21T12:54:55.423 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-03-21T12:54:55.423 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (11 ms) 2024-03-21T12:54:55.423 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (8518 ms total) 2024-03-21T12:54:55.424 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: 2024-03-21T12:54:55.424 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-03-21T12:54:55.424 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-03-21T12:54:55.424 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: watch_notify_test_cb 2024-03-21T12:54:55.424 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (926 ms) 2024-03-21T12:54:55.424 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (926 ms total) 2024-03-21T12:54:55.424 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: 2024-03-21T12:54:55.424 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-03-21T12:54:55.424 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (29639 ms total) 2024-03-21T12:54:55.424 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-03-21T12:54:55.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:55 smithi155 ceph-mon[33846]: pgmap v100: 752 pgs: 98 creating+peering, 654 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 4.0 KiB/s rd, 5.5 KiB/s wr, 4 op/s 2024-03-21T12:54:55.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:55 smithi155 ceph-mon[33846]: pgmap v101: 752 pgs: 98 creating+peering, 654 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 6.4 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2024-03-21T12:54:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:55 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:55 smithi155 ceph-mon[33846]: Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-21T12:54:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:55 smithi155 ceph-mon[33846]: from='client.25402 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82129-12"}]': finished 2024-03-21T12:54:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:55 smithi155 ceph-mon[33846]: from='client.25468 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-81984-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:55 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:55 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3564148337' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:55 smithi155 ceph-mon[33846]: osdmap e94: 8 total, 8 up, 8 in 2024-03-21T12:54:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:55 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/882562951' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:55 smithi155 ceph-mon[33846]: from='client.25501 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi043-82926-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:55 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:55 smithi155 ceph-mon[33846]: from='client.25402 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:55 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[30062]: pgmap v100: 752 pgs: 98 creating+peering, 654 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 4.0 KiB/s rd, 5.5 KiB/s wr, 4 op/s 2024-03-21T12:54:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[30062]: pgmap v101: 752 pgs: 98 creating+peering, 654 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 6.4 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2024-03-21T12:54:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[30062]: Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-21T12:54:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[30062]: from='client.25402 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82129-12"}]': finished 2024-03-21T12:54:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[30062]: from='client.25468 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-81984-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3564148337' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[30062]: osdmap e94: 8 total, 8 up, 8 in 2024-03-21T12:54:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/882562951' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:55.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[30062]: from='client.25501 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi043-82926-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:55.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:55.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[30062]: from='client.25402 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:55.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:55.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[41699]: pgmap v100: 752 pgs: 98 creating+peering, 654 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 4.0 KiB/s rd, 5.5 KiB/s wr, 4 op/s 2024-03-21T12:54:55.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[41699]: pgmap v101: 752 pgs: 98 creating+peering, 654 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 6.4 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2024-03-21T12:54:55.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:55.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[41699]: Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-21T12:54:55.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[41699]: from='client.25402 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi043-82129-12"}]': finished 2024-03-21T12:54:55.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[41699]: from='client.25468 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi043-81984-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:55.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:55.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3564148337' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:55.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[41699]: osdmap e94: 8 total, 8 up, 8 in 2024-03-21T12:54:55.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/882562951' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:55.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[41699]: from='client.25501 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi043-82926-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:54:55.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:55.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[41699]: from='client.25402 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82129-12"}]: dispatch 2024-03-21T12:54:55.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:55 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:56.462 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-03-21T12:54:56.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-03-21T12:54:56.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-03-21T12:54:56.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-03-21T12:54:56.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-03-21T12:54:56.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify 2024-03-21T12:54:56.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (3096 ms) 2024-03-21T12:54:56.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-03-21T12:54:56.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (7 ms) 2024-03-21T12:54:56.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (3104 ms total) 2024-03-21T12:54:56.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: 2024-03-21T12:54:56.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-03-21T12:54:56.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-03-21T12:54:56.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify 2024-03-21T12:54:56.463 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (116 ms) 2024-03-21T12:54:56.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-03-21T12:54:56.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify 2024-03-21T12:54:56.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (3986 ms) 2024-03-21T12:54:56.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-03-21T12:54:56.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (8 ms) 2024-03-21T12:54:56.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-03-21T12:54:56.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (3 ms) 2024-03-21T12:54:56.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-03-21T12:54:56.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 93961911045872 notify_id 399431958529 notifier_gid 35396 2024-03-21T12:54:56.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (5 ms) 2024-03-21T12:54:56.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-03-21T12:54:56.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 93961911045872 notify_id 399431958530 notifier_gid 35396 2024-03-21T12:54:56.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (5 ms) 2024-03-21T12:54:56.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-03-21T12:54:56.464 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 93961911045872 notify_id 399431958530 notifier_gid 35396 2024-03-21T12:54:56.465 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (5 ms) 2024-03-21T12:54:56.465 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-03-21T12:54:56.465 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 93961911045664 notify_id 399431958531 notifier_gid 35396 2024-03-21T12:54:56.465 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (4 ms) 2024-03-21T12:54:56.465 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-03-21T12:54:56.465 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 93961911046032 notify_id 399431958532 notifier_gid 35396 2024-03-21T12:54:56.465 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (5 ms) 2024-03-21T12:54:56.465 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-03-21T12:54:56.465 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 93961911046032 notify_id 399431958530 notifier_gid 35396 2024-03-21T12:54:56.465 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (4 ms) 2024-03-21T12:54:56.465 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-03-21T12:54:56.465 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: trying... 2024-03-21T12:54:56.465 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 93961911065136 notify_id 399431958533 notifier_gid 35396 2024-03-21T12:54:56.465 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: timed out 2024-03-21T12:54:56.465 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: flushing 2024-03-21T12:54:57.006 INFO:tasks.workunit.client.0.smithi043.stdout: api_watc api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-03-21T12:54:57.006 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [----------] Global test environment set-up. 2024-03-21T12:54:57.006 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-03-21T12:54:57.006 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-03-21T12:54:57.006 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5035 ms) 2024-03-21T12:54:57.006 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-03-21T12:54:57.006 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (1057 ms) 2024-03-21T12:54:57.006 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-03-21T12:54:57.006 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20024 ms) 2024-03-21T12:54:57.006 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-03-21T12:54:57.006 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: attempt 0 of 20 2024-03-21T12:54:57.006 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (5045 ms) 2024-03-21T12:54:57.007 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (31161 ms total) 2024-03-21T12:54:57.007 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: 2024-03-21T12:54:57.007 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [----------] Global test environment tear-down 2024-03-21T12:54:57.007 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (31161 ms total) 2024-03-21T12:54:57.007 INFO:tasks.workunit.client.0.smithi043.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-03-21T12:54:57.159 INFO:tasks.workunit.client.0.smithi043.stdout: cluster: 2024-03-21T12:54:57.159 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: id: c06c8d1a-e780-11ee-95cd-87774f69a715 2024-03-21T12:54:57.159 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: health: HEALTH_WARN 2024-03-21T12:54:57.159 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 32 pool(s) do not have an application enabled 2024-03-21T12:54:57.159 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-03-21T12:54:57.159 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: services: 2024-03-21T12:54:57.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-03-21T12:54:57.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: mgr: y(active, since 2m), standbys: x 2024-03-21T12:54:57.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: osd: 8 osds: 8 up (since 3m), 8 in (since 3m) 2024-03-21T12:54:57.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-03-21T12:54:57.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-03-21T12:54:57.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-03-21T12:54:57.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-03-21T12:54:57.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: data: 2024-03-21T12:54:57.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: pools: 42 pools, 1284 pgs 2024-03-21T12:54:57.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: objects: 198 objects, 582 KiB 2024-03-21T12:54:57.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: usage: 235 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:54:57.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: pgs: 35.748% pgs unknown 2024-03-21T12:54:57.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 53.972% pgs not active 2024-03-21T12:54:57.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 693 creating+peering 2024-03-21T12:54:57.160 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 459 unknown 2024-03-21T12:54:57.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 132 active+clean 2024-03-21T12:54:57.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-03-21T12:54:57.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: io: 2024-03-21T12:54:57.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: client: 1.2 KiB/s rd, 1 op/s rd, 0 op/s wr 2024-03-21T12:54:57.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-03-21T12:54:57.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-03-21T12:54:57.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ OK ] LibRadosService.StatusFormat (6225 ms) 2024-03-21T12:54:57.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ RUN ] LibRadosService.Status 2024-03-21T12:54:57.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ OK ] LibRadosService.Status (20037 ms) 2024-03-21T12:54:57.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [----------] 4 tests from LibRadosService (31341 ms total) 2024-03-21T12:54:57.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: 2024-03-21T12:54:57.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [----------] Global test environment tear-down 2024-03-21T12:54:57.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [==========] 4 tests from 1 test suite ran. (31341 ms total) 2024-03-21T12:54:57.161 INFO:tasks.workunit.client.0.smithi043.stdout: api_service: [ PASSED ] 4 tests. 2024-03-21T12:54:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:56 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/882562951' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:56 smithi155 ceph-mon[33846]: from='client.25501 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi043-82926-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:56 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:57.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:56 smithi155 ceph-mon[33846]: from='client.25402 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82129-12"}]': finished 2024-03-21T12:54:57.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:56 smithi155 ceph-mon[33846]: osdmap e95: 8 total, 8 up, 8 in 2024-03-21T12:54:57.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:56 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/468648503' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi043-82926-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi043-82926-3"}]: dispatch 2024-03-21T12:54:57.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:56 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/212817740' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:57.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:56 smithi155 ceph-mon[33846]: from='client.25501 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi043-82926-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi043-82926-3"}]: dispatch 2024-03-21T12:54:57.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:56 smithi155 ceph-mon[33846]: from='client.25498 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:57.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:56 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:56 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/882562951' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:56 smithi043 ceph-mon[41699]: from='client.25501 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi043-82926-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:56 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:56 smithi043 ceph-mon[41699]: from='client.25402 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82129-12"}]': finished 2024-03-21T12:54:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:56 smithi043 ceph-mon[41699]: osdmap e95: 8 total, 8 up, 8 in 2024-03-21T12:54:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:56 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/468648503' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi043-82926-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi043-82926-3"}]: dispatch 2024-03-21T12:54:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:56 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/212817740' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:56 smithi043 ceph-mon[41699]: from='client.25501 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi043-82926-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi043-82926-3"}]: dispatch 2024-03-21T12:54:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:56 smithi043 ceph-mon[41699]: from='client.25498 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:57.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:56 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:57.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:56 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/882562951' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:57.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:56 smithi043 ceph-mon[30062]: from='client.25501 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi043-82926-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:54:57.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:56 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:57.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:56 smithi043 ceph-mon[30062]: from='client.25402 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi043-82129-12"}]': finished 2024-03-21T12:54:57.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:56 smithi043 ceph-mon[30062]: osdmap e95: 8 total, 8 up, 8 in 2024-03-21T12:54:57.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:56 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/468648503' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi043-82926-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi043-82926-3"}]: dispatch 2024-03-21T12:54:57.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:56 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/212817740' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:57.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:56 smithi043 ceph-mon[30062]: from='client.25501 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi043-82926-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi043-82926-3"}]: dispatch 2024-03-21T12:54:57.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:56 smithi043 ceph-mon[30062]: from='client.25498 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:57.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:56 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:58.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:54:58 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:54:58.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:58 smithi155 ceph-mon[33846]: pgmap v104: 744 pgs: 160 unknown, 96 creating+peering, 488 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 4.0 KiB/s wr, 0 op/s 2024-03-21T12:54:58.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:58 smithi155 ceph-mon[33846]: from='client.25498 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:58.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:58 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:58.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:58 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1314858984' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-81636-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:58.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:58 smithi155 ceph-mon[33846]: osdmap e96: 8 total, 8 up, 8 in 2024-03-21T12:54:58.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:58 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:58.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:58 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3343053782' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-21T12:54:58.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:58 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:58.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:58 smithi155 ceph-mon[33846]: from='client.35516 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-81636-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:58.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:58 smithi155 ceph-mon[33846]: Health check update: 11 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:54:58.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:58 smithi155 ceph-mon[33846]: from='client.25501 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi043-82926-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi043-82926-3"}]': finished 2024-03-21T12:54:58.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:58 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:58.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:58 smithi155 ceph-mon[33846]: from='client.35516 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-81636-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:58.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:58 smithi155 ceph-mon[33846]: osdmap e97: 8 total, 8 up, 8 in 2024-03-21T12:54:58.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:58 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:54:58.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:54:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:54:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[30062]: pgmap v104: 744 pgs: 160 unknown, 96 creating+peering, 488 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 4.0 KiB/s wr, 0 op/s 2024-03-21T12:54:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[30062]: from='client.25498 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1314858984' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-81636-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[30062]: osdmap e96: 8 total, 8 up, 8 in 2024-03-21T12:54:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3343053782' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-21T12:54:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[30062]: from='client.35516 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-81636-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[30062]: Health check update: 11 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:54:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[30062]: from='client.25501 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi043-82926-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi043-82926-3"}]': finished 2024-03-21T12:54:58.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:58.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[30062]: from='client.35516 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-81636-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:58.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[30062]: osdmap e97: 8 total, 8 up, 8 in 2024-03-21T12:54:58.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:54:58.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[41699]: pgmap v104: 744 pgs: 160 unknown, 96 creating+peering, 488 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 4.0 KiB/s wr, 0 op/s 2024-03-21T12:54:58.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[41699]: from='client.25498 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:58.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:58.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1314858984' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-81636-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:58.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[41699]: osdmap e96: 8 total, 8 up, 8 in 2024-03-21T12:54:58.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:58.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3343053782' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-21T12:54:58.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:58.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[41699]: from='client.35516 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-81636-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:58.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[41699]: Health check update: 11 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:54:58.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[41699]: from='client.25501 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi043-82926-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi043-82926-3"}]': finished 2024-03-21T12:54:58.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:58.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[41699]: from='client.35516 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi043-81636-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:54:58.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[41699]: osdmap e97: 8 total, 8 up, 8 in 2024-03-21T12:54:58.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:58 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:54:59.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:59 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-21T12:54:59.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:59 smithi155 ceph-mon[33846]: pgmap v107: 776 pgs: 288 unknown, 488 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail 2024-03-21T12:54:59.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:59 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:59.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:59 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:54:59.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:59 smithi155 ceph-mon[33846]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-21T12:54:59.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:59 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-7", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:54:59.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:59 smithi155 ceph-mon[33846]: osdmap e98: 8 total, 8 up, 8 in 2024-03-21T12:54:59.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:59 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2877872190' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:59.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:59 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-7"}]: dispatch 2024-03-21T12:54:59.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:59 smithi155 ceph-mon[33846]: from='client.35540 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:59.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:59 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-7"}]: dispatch 2024-03-21T12:54:59.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:54:59 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:59 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-21T12:54:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:59 smithi043 ceph-mon[41699]: pgmap v107: 776 pgs: 288 unknown, 488 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail 2024-03-21T12:54:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:59 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:59 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:54:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:59 smithi043 ceph-mon[41699]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-21T12:54:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:59 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-7", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:54:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:59 smithi043 ceph-mon[41699]: osdmap e98: 8 total, 8 up, 8 in 2024-03-21T12:54:59.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:59 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2877872190' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:59.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:59 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-7"}]: dispatch 2024-03-21T12:54:59.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:59 smithi043 ceph-mon[41699]: from='client.35540 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:59.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:59 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-7"}]: dispatch 2024-03-21T12:54:59.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:54:59 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:59.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:59 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-21T12:54:59.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:59 smithi043 ceph-mon[30062]: pgmap v107: 776 pgs: 288 unknown, 488 active+clean; 145 MiB data, 783 MiB used, 714 GiB / 715 GiB avail 2024-03-21T12:54:59.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:59 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:54:59.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:59 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:54:59.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:59 smithi043 ceph-mon[30062]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-21T12:54:59.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:59 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-7", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:54:59.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:59 smithi043 ceph-mon[30062]: osdmap e98: 8 total, 8 up, 8 in 2024-03-21T12:54:59.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:59 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2877872190' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:59.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:59 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-7"}]: dispatch 2024-03-21T12:54:59.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:59 smithi043 ceph-mon[30062]: from='client.35540 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:54:59.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:59 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-7"}]: dispatch 2024-03-21T12:54:59.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:54:59 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:00.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:00 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:55:00.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:00 smithi155 ceph-mon[33846]: pgmap v110: 656 pgs: 8 creating+peering, 96 unknown, 552 active+clean; 145 MiB data, 776 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-21T12:55:00.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:00 smithi155 ceph-mon[33846]: from='client.35540 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:00 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-7"}]': finished 2024-03-21T12:55:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:00 smithi155 ceph-mon[33846]: osdmap e99: 8 total, 8 up, 8 in 2024-03-21T12:55:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:00 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3102440440' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:00 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3470209039' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-81636-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:00 smithi155 ceph-mon[33846]: from='client.25561 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:00 smithi155 ceph-mon[33846]: from='client.25564 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-81636-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:00 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:55:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:00 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:00 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:55:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:00 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:55:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:00 smithi043 ceph-mon[30062]: pgmap v110: 656 pgs: 8 creating+peering, 96 unknown, 552 active+clean; 145 MiB data, 776 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-21T12:55:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:00 smithi043 ceph-mon[30062]: from='client.35540 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:00 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-7"}]': finished 2024-03-21T12:55:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:00 smithi043 ceph-mon[30062]: osdmap e99: 8 total, 8 up, 8 in 2024-03-21T12:55:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:00 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3102440440' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:00 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3470209039' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-81636-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:00 smithi043 ceph-mon[30062]: from='client.25561 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:00 smithi043 ceph-mon[30062]: from='client.25564 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-81636-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:00 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:55:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:00 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:00.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:00 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:55:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:00 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:55:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:00 smithi043 ceph-mon[41699]: pgmap v110: 656 pgs: 8 creating+peering, 96 unknown, 552 active+clean; 145 MiB data, 776 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 6 op/s 2024-03-21T12:55:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:00 smithi043 ceph-mon[41699]: from='client.35540 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:00 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-7"}]': finished 2024-03-21T12:55:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:00 smithi043 ceph-mon[41699]: osdmap e99: 8 total, 8 up, 8 in 2024-03-21T12:55:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:00 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3102440440' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:00 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3470209039' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-81636-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:00 smithi043 ceph-mon[41699]: from='client.25561 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:00 smithi043 ceph-mon[41699]: from='client.25564 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-81636-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:00 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:55:00.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:00 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:00.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:00 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:55:01.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[41699]: from='client.25561 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:01.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[41699]: from='client.25564 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-81636-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:01.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:55:01.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[41699]: osdmap e100: 8 total, 8 up, 8 in 2024-03-21T12:55:01.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/468648503' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi043-82926-3"}]: dispatch 2024-03-21T12:55:01.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-7"}]: dispatch 2024-03-21T12:55:01.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/687977662' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-82022-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:01.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3176393426' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81661-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:01.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[41699]: from='client.25501 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi043-82926-3"}]: dispatch 2024-03-21T12:55:01.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-7"}]: dispatch 2024-03-21T12:55:01.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[41699]: from='client.35570 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-82022-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:01.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[41699]: from='client.35567 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81661-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:01.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:01.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[30062]: from='client.25561 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:01.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[30062]: from='client.25564 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-81636-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:01.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:55:01.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[30062]: osdmap e100: 8 total, 8 up, 8 in 2024-03-21T12:55:01.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/468648503' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi043-82926-3"}]: dispatch 2024-03-21T12:55:01.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-7"}]: dispatch 2024-03-21T12:55:01.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/687977662' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-82022-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:01.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3176393426' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81661-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:01.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[30062]: from='client.25501 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi043-82926-3"}]: dispatch 2024-03-21T12:55:01.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-7"}]: dispatch 2024-03-21T12:55:01.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[30062]: from='client.35570 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-82022-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:01.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[30062]: from='client.35567 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81661-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:01.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:01 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:01.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:01 smithi155 ceph-mon[33846]: from='client.25561 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:01.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:01 smithi155 ceph-mon[33846]: from='client.25564 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi043-81636-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:01.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:01 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:55:01.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:01 smithi155 ceph-mon[33846]: osdmap e100: 8 total, 8 up, 8 in 2024-03-21T12:55:01.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:01 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/468648503' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi043-82926-3"}]: dispatch 2024-03-21T12:55:01.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:01 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-7"}]: dispatch 2024-03-21T12:55:01.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:01 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/687977662' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-82022-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:01.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:01 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3176393426' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81661-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:01.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:01 smithi155 ceph-mon[33846]: from='client.25501 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi043-82926-3"}]: dispatch 2024-03-21T12:55:01.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:01 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-7"}]: dispatch 2024-03-21T12:55:01.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:01 smithi155 ceph-mon[33846]: from='client.35570 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-82022-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:01.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:01 smithi155 ceph-mon[33846]: from='client.35567 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81661-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:01.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:01 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:02.494 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: Running main() from gmock_main.cc 2024-03-21T12:55:02.494 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [==========] Running 3 tests from 1 test suite. 2024-03-21T12:55:02.494 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [----------] Global test environment set-up. 2024-03-21T12:55:02.494 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [----------] 3 tests from NeoradosECList 2024-03-21T12:55:02.494 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [ RUN ] NeoradosECList.ListObjects 2024-03-21T12:55:02.494 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [ OK ] NeoradosECList.ListObjects (19289 ms) 2024-03-21T12:55:02.494 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsNS 2024-03-21T12:55:02.494 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsNS (7988 ms) 2024-03-21T12:55:02.495 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsMany 2024-03-21T12:55:02.495 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsMany (9193 ms) 2024-03-21T12:55:02.495 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [----------] 3 tests from NeoradosECList (36470 ms total) 2024-03-21T12:55:02.495 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: 2024-03-21T12:55:02.495 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [----------] Global test environment tear-down 2024-03-21T12:55:02.495 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [==========] 3 tests from 1 test suite ran. (36470 ms total) 2024-03-21T12:55:02.495 INFO:tasks.workunit.client.0.smithi043.stdout: ec_list: [ PASSED ] 3 tests. 2024-03-21T12:55:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:02 smithi155 ceph-mon[33846]: pgmap v113: 680 pgs: 192 unknown, 488 active+clean; 145 MiB data, 776 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-21T12:55:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:02 smithi155 ceph-mon[33846]: from='client.25501 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi043-82926-3"}]': finished 2024-03-21T12:55:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:02 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-7"}]': finished 2024-03-21T12:55:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:02 smithi155 ceph-mon[33846]: from='client.35570 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-82022-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:02 smithi155 ceph-mon[33846]: from='client.35567 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81661-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:02 smithi155 ceph-mon[33846]: osdmap e101: 8 total, 8 up, 8 in 2024-03-21T12:55:03.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:02 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3586487496' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:03.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:02 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/468648503' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi043-82926-3"}]: dispatch 2024-03-21T12:55:03.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:02 smithi155 ceph-mon[33846]: from='client.35582 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:03.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:02 smithi155 ceph-mon[33846]: from='client.25501 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi043-82926-3"}]: dispatch 2024-03-21T12:55:03.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:02 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:03.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:02 smithi155 ceph-mon[33846]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:03.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:02 smithi043 ceph-mon[30062]: pgmap v113: 680 pgs: 192 unknown, 488 active+clean; 145 MiB data, 776 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-21T12:55:03.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:02 smithi043 ceph-mon[30062]: from='client.25501 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi043-82926-3"}]': finished 2024-03-21T12:55:03.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:02 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-7"}]': finished 2024-03-21T12:55:03.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:02 smithi043 ceph-mon[30062]: from='client.35570 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-82022-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:03.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:02 smithi043 ceph-mon[30062]: from='client.35567 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81661-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:03.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:02 smithi043 ceph-mon[30062]: osdmap e101: 8 total, 8 up, 8 in 2024-03-21T12:55:03.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:02 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3586487496' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:03.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:02 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/468648503' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi043-82926-3"}]: dispatch 2024-03-21T12:55:03.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:02 smithi043 ceph-mon[30062]: from='client.35582 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:03.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:02 smithi043 ceph-mon[30062]: from='client.25501 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi043-82926-3"}]: dispatch 2024-03-21T12:55:03.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:02 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:03.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:02 smithi043 ceph-mon[30062]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:03.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:02 smithi043 ceph-mon[41699]: pgmap v113: 680 pgs: 192 unknown, 488 active+clean; 145 MiB data, 776 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-21T12:55:03.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:02 smithi043 ceph-mon[41699]: from='client.25501 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi043-82926-3"}]': finished 2024-03-21T12:55:03.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:02 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-7"}]': finished 2024-03-21T12:55:03.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:02 smithi043 ceph-mon[41699]: from='client.35570 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi043-82022-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:03.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:02 smithi043 ceph-mon[41699]: from='client.35567 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81661-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:03.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:02 smithi043 ceph-mon[41699]: osdmap e101: 8 total, 8 up, 8 in 2024-03-21T12:55:03.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:02 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3586487496' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:03.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:02 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/468648503' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi043-82926-3"}]: dispatch 2024-03-21T12:55:03.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:02 smithi043 ceph-mon[41699]: from='client.35582 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:03.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:02 smithi043 ceph-mon[41699]: from='client.25501 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi043-82926-3"}]: dispatch 2024-03-21T12:55:03.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:02 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:03.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:02 smithi043 ceph-mon[41699]: Health check update: 9 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:04.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[30062]: from='client.35582 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:04.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[30062]: from='client.25501 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi043-82926-3"}]': finished 2024-03-21T12:55:04.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[30062]: osdmap e102: 8 total, 8 up, 8 in 2024-03-21T12:55:04.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:04.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[30062]: pgmap v116: 648 pgs: 32 creating+peering, 616 active+clean; 145 MiB data, 769 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-03-21T12:55:04.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[30062]: osdmap e103: 8 total, 8 up, 8 in 2024-03-21T12:55:04.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/643758892' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81661-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:04.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3633515712' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-81636-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:04.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:04.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:55:04.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:55:04.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[30062]: from='client.25606 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81661-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:04.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:04.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[30062]: from='client.25603 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-81636-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:04.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:04.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[41699]: from='client.35582 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:04.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[41699]: from='client.25501 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi043-82926-3"}]': finished 2024-03-21T12:55:04.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[41699]: osdmap e102: 8 total, 8 up, 8 in 2024-03-21T12:55:04.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:04.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[41699]: pgmap v116: 648 pgs: 32 creating+peering, 616 active+clean; 145 MiB data, 769 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-03-21T12:55:04.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[41699]: osdmap e103: 8 total, 8 up, 8 in 2024-03-21T12:55:04.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/643758892' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81661-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:04.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3633515712' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-81636-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:04.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:04.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:55:04.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:55:04.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[41699]: from='client.25606 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81661-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:04.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:04.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[41699]: from='client.25603 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-81636-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:04.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:04.661 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: Running main() from gmock_main.cc 2024-03-21T12:55:04.661 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [==========] Running 7 tests from 1 test suite. 2024-03-21T12:55:04.661 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [----------] Global test environment set-up. 2024-03-21T12:55:04.661 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps 2024-03-21T12:55:04.661 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertExists 2024-03-21T12:55:04.662 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertExists (11174 ms) 2024-03-21T12:55:04.662 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertVersion 2024-03-21T12:55:04.662 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertVersion (6974 ms) 2024-03-21T12:55:04.662 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Xattrs 2024-03-21T12:55:04.662 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ OK ] NeoRadosWriteOps.Xattrs (5127 ms) 2024-03-21T12:55:04.662 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Write 2024-03-21T12:55:04.662 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ OK ] NeoRadosWriteOps.Write (3964 ms) 2024-03-21T12:55:04.662 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Exec 2024-03-21T12:55:04.662 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ OK ] NeoRadosWriteOps.Exec (3993 ms) 2024-03-21T12:55:04.662 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ RUN ] NeoRadosWriteOps.WriteSame 2024-03-21T12:55:04.662 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ OK ] NeoRadosWriteOps.WriteSame (3144 ms) 2024-03-21T12:55:04.662 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ RUN ] NeoRadosWriteOps.CmpExt 2024-03-21T12:55:04.662 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ OK ] NeoRadosWriteOps.CmpExt (4215 ms) 2024-03-21T12:55:04.663 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps (38591 ms total) 2024-03-21T12:55:04.663 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: 2024-03-21T12:55:04.663 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [----------] Global test environment tear-down 2024-03-21T12:55:04.663 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [==========] 7 tests from 1 test suite ran. (38591 ms total) 2024-03-21T12:55:04.663 INFO:tasks.workunit.client.0.smithi043.stdout: write_operations: [ PASSED ] 7 tests. 2024-03-21T12:55:04.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-03-21T12:55:04.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-03-21T12:55:04.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-03-21T12:55:04.665 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-03-21T12:55:04.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-03-21T12:55:04.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-03-21T12:55:04.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-03-21T12:55:04.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: 2024-03-21T12:55:04.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-03-21T12:55:04.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-03-21T12:55:04.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (11391 ms) 2024-03-21T12:55:04.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-03-21T12:55:04.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (6993 ms) 2024-03-21T12:55:04.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-03-21T12:55:04.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (5109 ms) 2024-03-21T12:55:04.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-03-21T12:55:04.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (3963 ms) 2024-03-21T12:55:04.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-03-21T12:55:04.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (3996 ms) 2024-03-21T12:55:04.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-03-21T12:55:04.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (3131 ms) 2024-03-21T12:55:04.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-03-21T12:55:04.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (4229 ms) 2024-03-21T12:55:04.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (38812 ms total) 2024-03-21T12:55:04.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: 2024-03-21T12:55:04.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-03-21T12:55:04.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (38812 ms total) 2024-03-21T12:55:04.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-03-21T12:55:04.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:04 smithi155 ceph-mon[33846]: from='client.35582 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82349-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:04.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:04 smithi155 ceph-mon[33846]: from='client.25501 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi043-82926-3"}]': finished 2024-03-21T12:55:04.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:04 smithi155 ceph-mon[33846]: osdmap e102: 8 total, 8 up, 8 in 2024-03-21T12:55:04.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:04.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:04 smithi155 ceph-mon[33846]: pgmap v116: 648 pgs: 32 creating+peering, 616 active+clean; 145 MiB data, 769 MiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 767 B/s wr, 4 op/s 2024-03-21T12:55:04.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:04 smithi155 ceph-mon[33846]: osdmap e103: 8 total, 8 up, 8 in 2024-03-21T12:55:04.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/643758892' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81661-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:04.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3633515712' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-81636-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:04.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:04.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:04 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:55:04.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:04 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:55:04.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:04 smithi155 ceph-mon[33846]: from='client.25606 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81661-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:04.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:04 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:04.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:04 smithi155 ceph-mon[33846]: from='client.25603 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-81636-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:04.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:05.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:05 smithi155 ceph-mon[33846]: from='client.25606 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81661-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:05.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:05 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:05.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:05 smithi155 ceph-mon[33846]: from='client.25603 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-81636-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:05.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:05 smithi155 ceph-mon[33846]: osdmap e104: 8 total, 8 up, 8 in 2024-03-21T12:55:05.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:05 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2009710192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:05.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:05 smithi155 ceph-mon[33846]: from='client.35624 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:05.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:05 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:55:05.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:05 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:05.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:05 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:55:05.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:05 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1869762951' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4"}]: dispatch 2024-03-21T12:55:05.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:05 smithi155 ceph-mon[33846]: from='client.35396 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4"}]: dispatch 2024-03-21T12:55:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:05 smithi043 ceph-mon[41699]: from='client.25606 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81661-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:05 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:05 smithi043 ceph-mon[41699]: from='client.25603 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-81636-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:05 smithi043 ceph-mon[41699]: osdmap e104: 8 total, 8 up, 8 in 2024-03-21T12:55:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:05 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2009710192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:05 smithi043 ceph-mon[41699]: from='client.35624 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:05 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:55:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:05 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:05 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:55:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:05 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1869762951' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4"}]: dispatch 2024-03-21T12:55:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:05 smithi043 ceph-mon[41699]: from='client.35396 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4"}]: dispatch 2024-03-21T12:55:05.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:05 smithi043 ceph-mon[30062]: from='client.25606 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi043-81661-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:05.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:05 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:05.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:05 smithi043 ceph-mon[30062]: from='client.25603 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi043-81636-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:05.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:05 smithi043 ceph-mon[30062]: osdmap e104: 8 total, 8 up, 8 in 2024-03-21T12:55:05.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:05 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2009710192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:05.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:05 smithi043 ceph-mon[30062]: from='client.35624 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:05.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:05 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:55:05.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:05 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:05.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:05 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:55:05.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:05 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1869762951' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4"}]: dispatch 2024-03-21T12:55:05.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:05 smithi043 ceph-mon[30062]: from='client.35396 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4"}]: dispatch 2024-03-21T12:55:06.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:06 smithi043 ceph-mon[30062]: pgmap v119: 680 pgs: 192 unknown, 488 active+clean; 145 MiB data, 769 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-21T12:55:06.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:06 smithi043 ceph-mon[30062]: from='client.35624 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:06.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:06 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-9", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:55:06.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:06 smithi043 ceph-mon[30062]: from='client.35396 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4"}]': finished 2024-03-21T12:55:06.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:06 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-9"}]: dispatch 2024-03-21T12:55:06.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:06 smithi043 ceph-mon[30062]: osdmap e105: 8 total, 8 up, 8 in 2024-03-21T12:55:06.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:06 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1869762951' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "tierpool": "test-rados-api-smithi043-82308-6"}]: dispatch 2024-03-21T12:55:06.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:06 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-9"}]: dispatch 2024-03-21T12:55:06.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:06 smithi043 ceph-mon[30062]: from='client.35396 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "tierpool": "test-rados-api-smithi043-82308-6"}]: dispatch 2024-03-21T12:55:06.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:06 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:06.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:06 smithi043 ceph-mon[41699]: pgmap v119: 680 pgs: 192 unknown, 488 active+clean; 145 MiB data, 769 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-21T12:55:06.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:06 smithi043 ceph-mon[41699]: from='client.35624 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:06.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:06 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-9", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:55:06.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:06 smithi043 ceph-mon[41699]: from='client.35396 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4"}]': finished 2024-03-21T12:55:06.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:06 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-9"}]: dispatch 2024-03-21T12:55:06.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:06 smithi043 ceph-mon[41699]: osdmap e105: 8 total, 8 up, 8 in 2024-03-21T12:55:06.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:06 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1869762951' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "tierpool": "test-rados-api-smithi043-82308-6"}]: dispatch 2024-03-21T12:55:06.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:06 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-9"}]: dispatch 2024-03-21T12:55:06.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:06 smithi043 ceph-mon[41699]: from='client.35396 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "tierpool": "test-rados-api-smithi043-82308-6"}]: dispatch 2024-03-21T12:55:06.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:06 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:07.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:06 smithi155 ceph-mon[33846]: pgmap v119: 680 pgs: 192 unknown, 488 active+clean; 145 MiB data, 769 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-21T12:55:07.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:06 smithi155 ceph-mon[33846]: from='client.35624 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:07.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:06 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-9", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:55:07.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:06 smithi155 ceph-mon[33846]: from='client.35396 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4"}]': finished 2024-03-21T12:55:07.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:06 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-9"}]: dispatch 2024-03-21T12:55:07.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:06 smithi155 ceph-mon[33846]: osdmap e105: 8 total, 8 up, 8 in 2024-03-21T12:55:07.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:06 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1869762951' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "tierpool": "test-rados-api-smithi043-82308-6"}]: dispatch 2024-03-21T12:55:07.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:06 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-9"}]: dispatch 2024-03-21T12:55:07.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:06 smithi155 ceph-mon[33846]: from='client.35396 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "tierpool": "test-rados-api-smithi043-82308-6"}]: dispatch 2024-03-21T12:55:07.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:06 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:07 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:55:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:07 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-9"}]': finished 2024-03-21T12:55:08.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:07 smithi155 ceph-mon[33846]: from='client.35396 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "tierpool": "test-rados-api-smithi043-82308-6"}]': finished 2024-03-21T12:55:08.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:07 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-9", "mode": "writeback"}]: dispatch 2024-03-21T12:55:08.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:07 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1869762951' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi043-82308-6", "pool2": "test-rados-api-smithi043-82308-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T12:55:08.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:07 smithi155 ceph-mon[33846]: osdmap e106: 8 total, 8 up, 8 in 2024-03-21T12:55:08.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:07 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-9", "mode": "writeback"}]: dispatch 2024-03-21T12:55:08.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:07 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/254185577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-81661-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:08.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:07 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/595148388' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-81636-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:08.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:07 smithi155 ceph-mon[33846]: from='client.35396 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi043-82308-6", "pool2": "test-rados-api-smithi043-82308-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T12:55:08.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:07 smithi155 ceph-mon[33846]: from='client.25630 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-81661-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:08.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:07 smithi155 ceph-mon[33846]: from='client.25633 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-81636-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:08.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:07 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:08.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:07 smithi155 ceph-mon[33846]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:08.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:07 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:55:08.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:07 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-9", "mode": "writeback"}]': finished 2024-03-21T12:55:08.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:07 smithi155 ceph-mon[33846]: from='client.35396 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi043-82308-6", "pool2": "test-rados-api-smithi043-82308-6", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T12:55:08.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:07 smithi155 ceph-mon[33846]: from='client.25630 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-81661-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:08.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:07 smithi155 ceph-mon[33846]: from='client.25633 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-81636-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:08.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:07 smithi155 ceph-mon[33846]: osdmap e107: 8 total, 8 up, 8 in 2024-03-21T12:55:08.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:07 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:55:08.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:55:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:55:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:55:08.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:55:08.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-9"}]': finished 2024-03-21T12:55:08.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[30062]: from='client.35396 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "tierpool": "test-rados-api-smithi043-82308-6"}]': finished 2024-03-21T12:55:08.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-9", "mode": "writeback"}]: dispatch 2024-03-21T12:55:08.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1869762951' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi043-82308-6", "pool2": "test-rados-api-smithi043-82308-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T12:55:08.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[30062]: osdmap e106: 8 total, 8 up, 8 in 2024-03-21T12:55:08.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-9", "mode": "writeback"}]: dispatch 2024-03-21T12:55:08.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/254185577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-81661-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:08.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/595148388' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-81636-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:08.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[30062]: from='client.35396 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi043-82308-6", "pool2": "test-rados-api-smithi043-82308-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T12:55:08.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[30062]: from='client.25630 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-81661-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:08.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[30062]: from='client.25633 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-81636-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:08.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:08.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[30062]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:08.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:55:08.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-9", "mode": "writeback"}]': finished 2024-03-21T12:55:08.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[30062]: from='client.35396 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi043-82308-6", "pool2": "test-rados-api-smithi043-82308-6", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T12:55:08.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[30062]: from='client.25630 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-81661-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:08.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[30062]: from='client.25633 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-81636-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:08.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[30062]: osdmap e107: 8 total, 8 up, 8 in 2024-03-21T12:55:08.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:55:08.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:55:08.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-9"}]': finished 2024-03-21T12:55:08.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[41699]: from='client.35396 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi043-82308-4", "tierpool": "test-rados-api-smithi043-82308-6"}]': finished 2024-03-21T12:55:08.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-9", "mode": "writeback"}]: dispatch 2024-03-21T12:55:08.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1869762951' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi043-82308-6", "pool2": "test-rados-api-smithi043-82308-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T12:55:08.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[41699]: osdmap e106: 8 total, 8 up, 8 in 2024-03-21T12:55:08.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-9", "mode": "writeback"}]: dispatch 2024-03-21T12:55:08.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/254185577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-81661-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:08.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/595148388' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-81636-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:08.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[41699]: from='client.35396 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi043-82308-6", "pool2": "test-rados-api-smithi043-82308-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T12:55:08.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[41699]: from='client.25630 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-81661-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:08.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[41699]: from='client.25633 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-81636-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:08.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:08.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[41699]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:08.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:55:08.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-9", "mode": "writeback"}]': finished 2024-03-21T12:55:08.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[41699]: from='client.35396 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi043-82308-6", "pool2": "test-rados-api-smithi043-82308-6", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T12:55:08.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[41699]: from='client.25630 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi043-81661-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:08.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[41699]: from='client.25633 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi043-81636-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:08.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[41699]: osdmap e107: 8 total, 8 up, 8 in 2024-03-21T12:55:08.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:07 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:55:08.303 INFO:tasks.workunit.client.0.smithi043.stdout:h_notify_pp: flushed 2024-03-21T12:55:08.303 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3004 ms) 2024-03-21T12:55:08.303 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-03-21T12:55:08.303 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: trying... 2024-03-21T12:55:08.303 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 93961911065136 notify_id 408021893123 notifier_gid 35396 2024-03-21T12:55:08.303 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: timed out 2024-03-21T12:55:08.303 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: flushing 2024-03-21T12:55:08.303 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: flushed 2024-03-21T12:55:08.303 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3008 ms) 2024-03-21T12:55:08.303 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-03-21T12:55:08.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: List watches 2024-03-21T12:55:08.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify2 2024-03-21T12:55:08.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 93961911046032 notify_id 425201762308 notifier_gid 35396 2024-03-21T12:55:08.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify2 done 2024-03-21T12:55:08.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: watch_check 2024-03-21T12:55:08.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: unwatch2 2024-03-21T12:55:08.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: flushing 2024-03-21T12:55:08.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: done 2024-03-21T12:55:08.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3093 ms) 2024-03-21T12:55:08.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-03-21T12:55:08.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: List watches 2024-03-21T12:55:08.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify2 2024-03-21T12:55:08.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: handle_notify cookie 93961911046032 notify_id 438086664195 notifier_gid 35396 2024-03-21T12:55:08.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: notify2 done 2024-03-21T12:55:08.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: watch_check 2024-03-21T12:55:08.304 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: unwatch2 2024-03-21T12:55:08.305 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: flushing 2024-03-21T12:55:08.305 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: done 2024-03-21T12:55:08.305 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3023 ms) 2024-03-21T12:55:08.305 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (16269 ms total) 2024-03-21T12:55:08.305 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: 2024-03-21T12:55:08.305 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-03-21T12:55:08.305 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (42479 ms total) 2024-03-21T12:55:08.305 INFO:tasks.workunit.client.0.smithi043.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-03-21T12:55:08.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:55:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:55:09.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:09 smithi155 ceph-mon[33846]: pgmap v122: 744 pgs: 256 unknown, 488 active+clean; 145 MiB data, 769 MiB used, 714 GiB / 715 GiB avail 2024-03-21T12:55:09.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:09 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:09.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:09 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:55:09.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:09 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:55:09.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:09 smithi155 ceph-mon[33846]: osdmap e108: 8 total, 8 up, 8 in 2024-03-21T12:55:09.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:09 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-9"}]: dispatch 2024-03-21T12:55:09.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:09 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-9"}]: dispatch 2024-03-21T12:55:09.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:09 smithi043 ceph-mon[30062]: pgmap v122: 744 pgs: 256 unknown, 488 active+clean; 145 MiB data, 769 MiB used, 714 GiB / 715 GiB avail 2024-03-21T12:55:09.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:09 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:09.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:09 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:55:09.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:09 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:55:09.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:09 smithi043 ceph-mon[30062]: osdmap e108: 8 total, 8 up, 8 in 2024-03-21T12:55:09.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:09 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-9"}]: dispatch 2024-03-21T12:55:09.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:09 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-9"}]: dispatch 2024-03-21T12:55:09.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:09 smithi043 ceph-mon[41699]: pgmap v122: 744 pgs: 256 unknown, 488 active+clean; 145 MiB data, 769 MiB used, 714 GiB / 715 GiB avail 2024-03-21T12:55:09.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:09 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:09.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:09 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:55:09.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:09 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:55:09.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:09 smithi043 ceph-mon[41699]: osdmap e108: 8 total, 8 up, 8 in 2024-03-21T12:55:09.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:09 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-9"}]: dispatch 2024-03-21T12:55:09.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:09 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-9"}]: dispatch 2024-03-21T12:55:10.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:55:10.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:10.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-03-21T12:55:10.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-03-21T12:55:10.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-03-21T12:55:10.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-03-21T12:55:10.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-03-21T12:55:10.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-03-21T12:55:10.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-03-21T12:55:10.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-03-21T12:55:10.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-03-21T12:55:10.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-03-21T12:55:10.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-03-21T12:55:10.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-03-21T12:55:10.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-03-21T12:55:10.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-03-21T12:55:10.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-03-21T12:55:10.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-03-21T12:55:10.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-03-21T12:55:10.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-03-21T12:55:10.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-03-21T12:55:10.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-03-21T12:55:10.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: 18.6 deep-scrub starts 2024-03-21T12:55:10.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: 18.6 deep-scrub ok 2024-03-21T12:55:10.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: pgmap v125: 548 pgs: 64 unknown, 484 active+clean; 145 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 op/s 2024-03-21T12:55:10.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: 18.9 deep-scrub starts 2024-03-21T12:55:10.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: 18.9 deep-scrub ok 2024-03-21T12:55:10.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:55:10.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-9"}]': finished 2024-03-21T12:55:10.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: osdmap e109: 8 total, 8 up, 8 in 2024-03-21T12:55:10.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2884148360' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-81636-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:10.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1926930581' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-81661-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:10.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1562083687' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:10.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='client.35681 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-81661-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:10.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='client.35678 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:10.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: 18.3 deep-scrub starts 2024-03-21T12:55:10.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: 18.3 deep-scrub ok 2024-03-21T12:55:10.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: 18.8 deep-scrub starts 2024-03-21T12:55:10.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: 18.8 deep-scrub ok 2024-03-21T12:55:10.711 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:10.711 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: 18.2 deep-scrub starts 2024-03-21T12:55:10.711 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:10 smithi155 ceph-mon[33846]: 18.2 deep-scrub ok 2024-03-21T12:55:10.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:55:10.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:10.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-03-21T12:55:10.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-03-21T12:55:10.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-03-21T12:55:10.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-03-21T12:55:10.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-03-21T12:55:10.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-03-21T12:55:10.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-03-21T12:55:10.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-03-21T12:55:10.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-03-21T12:55:10.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-03-21T12:55:10.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-03-21T12:55:10.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-03-21T12:55:10.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-03-21T12:55:10.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-03-21T12:55:10.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-03-21T12:55:10.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-03-21T12:55:10.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-03-21T12:55:10.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-03-21T12:55:10.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-03-21T12:55:10.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-03-21T12:55:10.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: 18.6 deep-scrub starts 2024-03-21T12:55:10.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: 18.6 deep-scrub ok 2024-03-21T12:55:10.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: pgmap v125: 548 pgs: 64 unknown, 484 active+clean; 145 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 op/s 2024-03-21T12:55:10.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: 18.9 deep-scrub starts 2024-03-21T12:55:10.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: 18.9 deep-scrub ok 2024-03-21T12:55:10.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:55:10.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-9"}]': finished 2024-03-21T12:55:10.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: osdmap e109: 8 total, 8 up, 8 in 2024-03-21T12:55:10.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2884148360' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-81636-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:10.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1926930581' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-81661-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:10.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1562083687' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:10.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='client.35681 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-81661-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:10.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='client.35678 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:10.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: 18.3 deep-scrub starts 2024-03-21T12:55:10.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: 18.3 deep-scrub ok 2024-03-21T12:55:10.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: 18.8 deep-scrub starts 2024-03-21T12:55:10.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: 18.8 deep-scrub ok 2024-03-21T12:55:10.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:10.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: 18.2 deep-scrub starts 2024-03-21T12:55:10.737 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[30062]: 18.2 deep-scrub ok 2024-03-21T12:55:10.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:55:10.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:10.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-03-21T12:55:10.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-03-21T12:55:10.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-03-21T12:55:10.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-03-21T12:55:10.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-03-21T12:55:10.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-03-21T12:55:10.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-03-21T12:55:10.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-03-21T12:55:10.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-03-21T12:55:10.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-03-21T12:55:10.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-03-21T12:55:10.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-03-21T12:55:10.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-03-21T12:55:10.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-03-21T12:55:10.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-03-21T12:55:10.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-03-21T12:55:10.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-03-21T12:55:10.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-03-21T12:55:10.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-03-21T12:55:10.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.155:0/2005155134' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-03-21T12:55:10.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: 18.6 deep-scrub starts 2024-03-21T12:55:10.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: 18.6 deep-scrub ok 2024-03-21T12:55:10.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: pgmap v125: 548 pgs: 64 unknown, 484 active+clean; 145 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 op/s 2024-03-21T12:55:10.740 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: 18.9 deep-scrub starts 2024-03-21T12:55:10.740 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: 18.9 deep-scrub ok 2024-03-21T12:55:10.740 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:55:10.740 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-9"}]': finished 2024-03-21T12:55:10.740 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: osdmap e109: 8 total, 8 up, 8 in 2024-03-21T12:55:10.740 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2884148360' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-81636-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:10.740 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1926930581' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-81661-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:10.740 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1562083687' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:10.740 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='client.35681 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-81661-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:10.740 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='client.35678 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:10.740 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: 18.3 deep-scrub starts 2024-03-21T12:55:10.740 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: 18.3 deep-scrub ok 2024-03-21T12:55:10.740 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: 18.8 deep-scrub starts 2024-03-21T12:55:10.740 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: 18.8 deep-scrub ok 2024-03-21T12:55:10.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:10.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: 18.2 deep-scrub starts 2024-03-21T12:55:10.741 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:10 smithi043 ceph-mon[41699]: 18.2 deep-scrub ok 2024-03-21T12:55:11.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:11 smithi043 ceph-mon[30062]: 18.7 deep-scrub starts 2024-03-21T12:55:11.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:11 smithi043 ceph-mon[30062]: 18.7 deep-scrub ok 2024-03-21T12:55:11.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:11 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2884148360' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-81636-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:11.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:11 smithi043 ceph-mon[30062]: from='client.35681 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-81661-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:11.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:11 smithi043 ceph-mon[30062]: from='client.35678 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:11.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:11 smithi043 ceph-mon[30062]: osdmap e110: 8 total, 8 up, 8 in 2024-03-21T12:55:11.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:11 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:11.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:11 smithi043 ceph-mon[41699]: 18.7 deep-scrub starts 2024-03-21T12:55:11.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:11 smithi043 ceph-mon[41699]: 18.7 deep-scrub ok 2024-03-21T12:55:11.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:11 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2884148360' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-81636-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:11.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:11 smithi043 ceph-mon[41699]: from='client.35681 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-81661-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:11.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:11 smithi043 ceph-mon[41699]: from='client.35678 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:11.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:11 smithi043 ceph-mon[41699]: osdmap e110: 8 total, 8 up, 8 in 2024-03-21T12:55:11.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:11 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:11 smithi155 ceph-mon[33846]: 18.7 deep-scrub starts 2024-03-21T12:55:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:11 smithi155 ceph-mon[33846]: 18.7 deep-scrub ok 2024-03-21T12:55:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:11 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2884148360' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi043-81636-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:11 smithi155 ceph-mon[33846]: from='client.35681 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi043-81661-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:11 smithi155 ceph-mon[33846]: from='client.35678 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81994-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:11 smithi155 ceph-mon[33846]: osdmap e110: 8 total, 8 up, 8 in 2024-03-21T12:55:11.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:11 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:12 smithi155 ceph-mon[33846]: 18.1 deep-scrub starts 2024-03-21T12:55:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:12 smithi155 ceph-mon[33846]: 18.1 deep-scrub ok 2024-03-21T12:55:12.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:12 smithi155 ceph-mon[33846]: 18.0 deep-scrub starts 2024-03-21T12:55:12.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:12 smithi155 ceph-mon[33846]: 18.0 deep-scrub ok 2024-03-21T12:55:12.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:12 smithi155 ceph-mon[33846]: pgmap v128: 580 pgs: 128 unknown, 452 active+clean; 145 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 op/s 2024-03-21T12:55:12.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:12 smithi155 ceph-mon[33846]: osdmap e111: 8 total, 8 up, 8 in 2024-03-21T12:55:12.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:12 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1562083687' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81994-10", "tierpool":"test-rados-api-smithi043-81994-10-cache", "force_nonempty":""}]: dispatch 2024-03-21T12:55:12.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:12 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:12.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:12 smithi155 ceph-mon[33846]: from='client.35678 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81994-10", "tierpool":"test-rados-api-smithi043-81994-10-cache", "force_nonempty":""}]: dispatch 2024-03-21T12:55:12.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:12 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:12.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:12 smithi155 ceph-mon[33846]: 18.4 deep-scrub starts 2024-03-21T12:55:12.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:12 smithi155 ceph-mon[33846]: 18.4 deep-scrub ok 2024-03-21T12:55:12.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:12 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:12.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:12 smithi155 ceph-mon[33846]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:12.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:12 smithi155 ceph-mon[33846]: from='client.35678 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81994-10", "tierpool":"test-rados-api-smithi043-81994-10-cache", "force_nonempty":""}]': finished 2024-03-21T12:55:12.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:12 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:12.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:12 smithi155 ceph-mon[33846]: osdmap e112: 8 total, 8 up, 8 in 2024-03-21T12:55:12.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:12 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1562083687' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81994-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:12.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:12 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3217400047' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-81636-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:12.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:12 smithi155 ceph-mon[33846]: from='client.35678 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81994-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:12.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:12 smithi155 ceph-mon[33846]: from='client.35699 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-81636-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:12.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:12 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3434719767' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[30062]: 18.1 deep-scrub starts 2024-03-21T12:55:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[30062]: 18.1 deep-scrub ok 2024-03-21T12:55:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[30062]: 18.0 deep-scrub starts 2024-03-21T12:55:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[30062]: 18.0 deep-scrub ok 2024-03-21T12:55:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[30062]: pgmap v128: 580 pgs: 128 unknown, 452 active+clean; 145 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 op/s 2024-03-21T12:55:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[30062]: osdmap e111: 8 total, 8 up, 8 in 2024-03-21T12:55:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1562083687' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81994-10", "tierpool":"test-rados-api-smithi043-81994-10-cache", "force_nonempty":""}]: dispatch 2024-03-21T12:55:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:12.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[30062]: from='client.35678 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81994-10", "tierpool":"test-rados-api-smithi043-81994-10-cache", "force_nonempty":""}]: dispatch 2024-03-21T12:55:12.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:12.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[30062]: 18.4 deep-scrub starts 2024-03-21T12:55:12.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[30062]: 18.4 deep-scrub ok 2024-03-21T12:55:12.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:12.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[30062]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:12.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[30062]: from='client.35678 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81994-10", "tierpool":"test-rados-api-smithi043-81994-10-cache", "force_nonempty":""}]': finished 2024-03-21T12:55:12.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:12.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[30062]: osdmap e112: 8 total, 8 up, 8 in 2024-03-21T12:55:12.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1562083687' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81994-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:12.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3217400047' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-81636-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:12.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[30062]: from='client.35678 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81994-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:12.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[30062]: from='client.35699 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-81636-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:12.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3434719767' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:12.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[41699]: 18.1 deep-scrub starts 2024-03-21T12:55:12.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[41699]: 18.1 deep-scrub ok 2024-03-21T12:55:12.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[41699]: 18.0 deep-scrub starts 2024-03-21T12:55:12.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[41699]: 18.0 deep-scrub ok 2024-03-21T12:55:12.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[41699]: pgmap v128: 580 pgs: 128 unknown, 452 active+clean; 145 MiB data, 746 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 op/s 2024-03-21T12:55:12.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[41699]: osdmap e111: 8 total, 8 up, 8 in 2024-03-21T12:55:12.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1562083687' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81994-10", "tierpool":"test-rados-api-smithi043-81994-10-cache", "force_nonempty":""}]: dispatch 2024-03-21T12:55:12.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:12.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[41699]: from='client.35678 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81994-10", "tierpool":"test-rados-api-smithi043-81994-10-cache", "force_nonempty":""}]: dispatch 2024-03-21T12:55:12.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:12.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[41699]: 18.4 deep-scrub starts 2024-03-21T12:55:12.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[41699]: 18.4 deep-scrub ok 2024-03-21T12:55:12.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:12.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[41699]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:12.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[41699]: from='client.35678 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81994-10", "tierpool":"test-rados-api-smithi043-81994-10-cache", "force_nonempty":""}]': finished 2024-03-21T12:55:12.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:12.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[41699]: osdmap e112: 8 total, 8 up, 8 in 2024-03-21T12:55:12.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1562083687' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81994-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:12.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3217400047' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-81636-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:12.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[41699]: from='client.35678 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81994-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:12.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[41699]: from='client.35699 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-81636-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:12.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:12 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3434719767' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:13 smithi155 ceph-mon[33846]: from='client.25693 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:13 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3434719767' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:13 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:13 smithi155 ceph-mon[33846]: from='client.25693 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:13 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3434719767' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81984-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:55:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:13 smithi155 ceph-mon[33846]: from='client.25693 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81984-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:55:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:13 smithi155 ceph-mon[33846]: from='client.35678 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81994-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:13 smithi155 ceph-mon[33846]: from='client.35699 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-81636-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:13 smithi155 ceph-mon[33846]: from='client.25693 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81984-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:55:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:13 smithi155 ceph-mon[33846]: osdmap e113: 8 total, 8 up, 8 in 2024-03-21T12:55:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:13 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3434719767' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-81984-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:13 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3689212295' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-81661-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:13 smithi155 ceph-mon[33846]: from='client.25693 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-81984-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:13 smithi155 ceph-mon[33846]: from='client.25702 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-81661-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:13 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1562083687' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81994-10", "tierpool":"test-rados-api-smithi043-81994-10-cache"}]: dispatch 2024-03-21T12:55:13.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:13 smithi155 ceph-mon[33846]: from='client.35678 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81994-10", "tierpool":"test-rados-api-smithi043-81994-10-cache"}]: dispatch 2024-03-21T12:55:13.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:13 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:55:13.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:13 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:55:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[30062]: from='client.25693 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3434719767' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[30062]: from='client.25693 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3434719767' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81984-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:55:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[30062]: from='client.25693 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81984-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:55:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[30062]: from='client.35678 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81994-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[30062]: from='client.35699 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-81636-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[30062]: from='client.25693 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81984-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:55:13.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[30062]: osdmap e113: 8 total, 8 up, 8 in 2024-03-21T12:55:13.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3434719767' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-81984-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:13.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3689212295' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-81661-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:13.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[30062]: from='client.25693 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-81984-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:13.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[30062]: from='client.25702 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-81661-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:13.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1562083687' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81994-10", "tierpool":"test-rados-api-smithi043-81994-10-cache"}]: dispatch 2024-03-21T12:55:13.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[30062]: from='client.35678 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81994-10", "tierpool":"test-rados-api-smithi043-81994-10-cache"}]: dispatch 2024-03-21T12:55:13.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:55:13.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:55:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[41699]: from='client.25693 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3434719767' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:13.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[41699]: from='client.25693 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:13.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3434719767' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81984-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:55:13.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[41699]: from='client.25693 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81984-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:55:13.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[41699]: from='client.35678 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81994-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:13.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[41699]: from='client.35699 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi043-81636-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:13.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[41699]: from='client.25693 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81984-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:55:13.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[41699]: osdmap e113: 8 total, 8 up, 8 in 2024-03-21T12:55:13.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3434719767' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-81984-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:13.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3689212295' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-81661-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:13.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[41699]: from='client.25693 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-81984-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:13.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[41699]: from='client.25702 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-81661-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:13.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1562083687' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81994-10", "tierpool":"test-rados-api-smithi043-81994-10-cache"}]: dispatch 2024-03-21T12:55:13.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[41699]: from='client.35678 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81994-10", "tierpool":"test-rados-api-smithi043-81994-10-cache"}]: dispatch 2024-03-21T12:55:13.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:55:13.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:13 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:55:15.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:14 smithi155 ceph-mon[33846]: pgmap v131: 580 pgs: 32 creating+peering, 548 active+clean; 217 MiB data, 1015 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 18 MiB/s wr, 4 op/s 2024-03-21T12:55:15.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:14 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:15.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:14 smithi155 ceph-mon[33846]: from='client.25702 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-81661-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:15.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:14 smithi155 ceph-mon[33846]: from='client.35678 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81994-10", "tierpool":"test-rados-api-smithi043-81994-10-cache"}]': finished 2024-03-21T12:55:15.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:14 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-11", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:55:15.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:14 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-11"}]: dispatch 2024-03-21T12:55:15.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:14 smithi155 ceph-mon[33846]: osdmap e114: 8 total, 8 up, 8 in 2024-03-21T12:55:15.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:14 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-11"}]: dispatch 2024-03-21T12:55:15.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:14 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:15.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:14 smithi043 ceph-mon[30062]: pgmap v131: 580 pgs: 32 creating+peering, 548 active+clean; 217 MiB data, 1015 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 18 MiB/s wr, 4 op/s 2024-03-21T12:55:15.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:14 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:15.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:14 smithi043 ceph-mon[30062]: from='client.25702 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-81661-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:15.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:14 smithi043 ceph-mon[30062]: from='client.35678 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81994-10", "tierpool":"test-rados-api-smithi043-81994-10-cache"}]': finished 2024-03-21T12:55:15.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:14 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-11", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:55:15.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:14 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-11"}]: dispatch 2024-03-21T12:55:15.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:14 smithi043 ceph-mon[30062]: osdmap e114: 8 total, 8 up, 8 in 2024-03-21T12:55:15.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:14 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-11"}]: dispatch 2024-03-21T12:55:15.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:14 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:15.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:14 smithi043 ceph-mon[41699]: pgmap v131: 580 pgs: 32 creating+peering, 548 active+clean; 217 MiB data, 1015 MiB used, 714 GiB / 715 GiB avail; 511 B/s rd, 18 MiB/s wr, 4 op/s 2024-03-21T12:55:15.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:14 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:15.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:14 smithi043 ceph-mon[41699]: from='client.25702 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi043-81661-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:15.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:14 smithi043 ceph-mon[41699]: from='client.35678 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81994-10", "tierpool":"test-rados-api-smithi043-81994-10-cache"}]': finished 2024-03-21T12:55:15.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:14 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-11", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:55:15.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:14 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-11"}]: dispatch 2024-03-21T12:55:15.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:14 smithi043 ceph-mon[41699]: osdmap e114: 8 total, 8 up, 8 in 2024-03-21T12:55:15.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:14 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-11"}]: dispatch 2024-03-21T12:55:15.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:14 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:15.315 INFO:tasks.workunit.client.0.smithi043.stdout: misc: Running main() from gmock_main.cc 2024-03-21T12:55:15.316 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [==========] Running 12 tests from 1 test suite. 2024-03-21T12:55:15.316 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [----------] Global test environment set-up. 2024-03-21T12:55:15.316 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [----------] 12 tests from NeoRadosMisc 2024-03-21T12:55:15.316 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.Version 2024-03-21T12:55:15.316 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.Version (10484 ms) 2024-03-21T12:55:15.316 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.WaitOSDMap 2024-03-21T12:55:15.316 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.WaitOSDMap (4534 ms) 2024-03-21T12:55:15.316 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.LongName 2024-03-21T12:55:15.316 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.LongName (6081 ms) 2024-03-21T12:55:15.316 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.LongLocator 2024-03-21T12:55:15.316 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.LongLocator (4315 ms) 2024-03-21T12:55:15.316 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.LongNamespace 2024-03-21T12:55:15.316 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.LongNamespace (3948 ms) 2024-03-21T12:55:15.317 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.LongAttrName 2024-03-21T12:55:15.317 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.LongAttrName (2904 ms) 2024-03-21T12:55:15.317 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.Exec 2024-03-21T12:55:15.317 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.Exec (3189 ms) 2024-03-21T12:55:15.317 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.Operate1 2024-03-21T12:55:15.317 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.Operate1 (3160 ms) 2024-03-21T12:55:15.317 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.Operate2 2024-03-21T12:55:15.317 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.Operate2 (2643 ms) 2024-03-21T12:55:15.317 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.BigObject 2024-03-21T12:55:15.317 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.BigObject (3099 ms) 2024-03-21T12:55:15.317 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.BigAttr 2024-03-21T12:55:15.317 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.BigAttr (1906 ms) 2024-03-21T12:55:15.317 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ RUN ] NeoRadosMisc.WriteSame 2024-03-21T12:55:15.317 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ OK ] NeoRadosMisc.WriteSame (3008 ms) 2024-03-21T12:55:15.317 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [----------] 12 tests from NeoRadosMisc (49271 ms total) 2024-03-21T12:55:15.317 INFO:tasks.workunit.client.0.smithi043.stdout: misc: 2024-03-21T12:55:15.318 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [----------] Global test environment tear-down 2024-03-21T12:55:15.318 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [==========] 12 tests from 1 test suite ran. (49271 ms total) 2024-03-21T12:55:15.318 INFO:tasks.workunit.client.0.smithi043.stdout: misc: [ PASSED ] 12 tests. 2024-03-21T12:55:16.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: Running main() from gmock_main.cc 2024-03-21T12:55:16.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-03-21T12:55:16.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [----------] Global test environment set-up. 2024-03-21T12:55:16.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-03-21T12:55:16.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-03-21T12:55:16.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolList (11203 ms) 2024-03-21T12:55:16.315 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-03-21T12:55:16.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (6266 ms) 2024-03-21T12:55:16.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-03-21T12:55:16.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (3955 ms) 2024-03-21T12:55:16.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-03-21T12:55:16.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (3592 ms) 2024-03-21T12:55:16.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-03-21T12:55:16.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (2567 ms) 2024-03-21T12:55:16.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-03-21T12:55:16.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (5001 ms) 2024-03-21T12:55:16.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-03-21T12:55:16.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (5348 ms) 2024-03-21T12:55:16.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-03-21T12:55:16.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (4654 ms) 2024-03-21T12:55:16.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-03-21T12:55:16.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (8007 ms) 2024-03-21T12:55:16.316 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [----------] 9 tests from LibRadosPools (50593 ms total) 2024-03-21T12:55:16.317 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: 2024-03-21T12:55:16.317 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [----------] Global test environment tear-down 2024-03-21T12:55:16.317 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (50593 ms total) 2024-03-21T12:55:16.317 INFO:tasks.workunit.client.0.smithi043.stdout: api_pool: [ PASSED ] 9 tests. 2024-03-21T12:55:16.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:16 smithi155 ceph-mon[33846]: pgmap v134: 644 pgs: 128 unknown, 516 active+clean; 217 MiB data, 1015 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 18 MiB/s wr, 3 op/s 2024-03-21T12:55:16.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:16 smithi155 ceph-mon[33846]: from='client.25693 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-81984-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-81984-10"}]': finished 2024-03-21T12:55:16.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:16 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-11"}]': finished 2024-03-21T12:55:16.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:16 smithi155 ceph-mon[33846]: osdmap e115: 8 total, 8 up, 8 in 2024-03-21T12:55:16.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:16 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2359422791' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-81636-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:16.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:16 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-11", "mode": "writeback"}]: dispatch 2024-03-21T12:55:16.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:16 smithi155 ceph-mon[33846]: from='client.35726 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-81636-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:16.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:16 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-11", "mode": "writeback"}]: dispatch 2024-03-21T12:55:16.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:16 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:16.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:16 smithi043 ceph-mon[30062]: pgmap v134: 644 pgs: 128 unknown, 516 active+clean; 217 MiB data, 1015 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 18 MiB/s wr, 3 op/s 2024-03-21T12:55:16.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:16 smithi043 ceph-mon[30062]: from='client.25693 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-81984-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-81984-10"}]': finished 2024-03-21T12:55:16.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:16 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-11"}]': finished 2024-03-21T12:55:16.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:16 smithi043 ceph-mon[30062]: osdmap e115: 8 total, 8 up, 8 in 2024-03-21T12:55:16.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:16 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2359422791' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-81636-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:16.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:16 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-11", "mode": "writeback"}]: dispatch 2024-03-21T12:55:16.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:16 smithi043 ceph-mon[30062]: from='client.35726 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-81636-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:16.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:16 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-11", "mode": "writeback"}]: dispatch 2024-03-21T12:55:16.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:16 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:16.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:16 smithi043 ceph-mon[41699]: pgmap v134: 644 pgs: 128 unknown, 516 active+clean; 217 MiB data, 1015 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 18 MiB/s wr, 3 op/s 2024-03-21T12:55:16.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:16 smithi043 ceph-mon[41699]: from='client.25693 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi043-81984-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi043-81984-10"}]': finished 2024-03-21T12:55:16.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:16 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-11"}]': finished 2024-03-21T12:55:16.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:16 smithi043 ceph-mon[41699]: osdmap e115: 8 total, 8 up, 8 in 2024-03-21T12:55:16.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:16 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2359422791' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-81636-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:16.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:16 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-11", "mode": "writeback"}]: dispatch 2024-03-21T12:55:16.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:16 smithi043 ceph-mon[41699]: from='client.35726 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-81636-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:16.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:16 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-11", "mode": "writeback"}]: dispatch 2024-03-21T12:55:16.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:16 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:17.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:17 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:55:17.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:17 smithi155 ceph-mon[33846]: from='client.35726 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-81636-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:17.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:17 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-11", "mode": "writeback"}]': finished 2024-03-21T12:55:17.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:17 smithi155 ceph-mon[33846]: osdmap e116: 8 total, 8 up, 8 in 2024-03-21T12:55:17.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:17 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1384663714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81661-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:17.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:17 smithi155 ceph-mon[33846]: from='client.25726 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81661-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:17.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:17 smithi155 ceph-mon[33846]: 18.5 deep-scrub starts 2024-03-21T12:55:17.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:17 smithi155 ceph-mon[33846]: 18.5 deep-scrub ok 2024-03-21T12:55:17.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:17 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:17.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:17 smithi155 ceph-mon[33846]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:17.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:17 smithi155 ceph-mon[33846]: from='client.25726 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81661-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:17.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:17 smithi155 ceph-mon[33846]: osdmap e117: 8 total, 8 up, 8 in 2024-03-21T12:55:17.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:17 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:55:17.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:17 smithi043 ceph-mon[30062]: from='client.35726 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-81636-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:17.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:17 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-11", "mode": "writeback"}]': finished 2024-03-21T12:55:17.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:17 smithi043 ceph-mon[30062]: osdmap e116: 8 total, 8 up, 8 in 2024-03-21T12:55:17.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:17 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1384663714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81661-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:17.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:17 smithi043 ceph-mon[30062]: from='client.25726 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81661-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:17.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:17 smithi043 ceph-mon[30062]: 18.5 deep-scrub starts 2024-03-21T12:55:17.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:17 smithi043 ceph-mon[30062]: 18.5 deep-scrub ok 2024-03-21T12:55:17.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:17 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:17.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:17 smithi043 ceph-mon[30062]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:17.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:17 smithi043 ceph-mon[30062]: from='client.25726 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81661-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:17.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:17 smithi043 ceph-mon[30062]: osdmap e117: 8 total, 8 up, 8 in 2024-03-21T12:55:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:17 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:55:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:17 smithi043 ceph-mon[41699]: from='client.35726 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi043-81636-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:17 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-11", "mode": "writeback"}]': finished 2024-03-21T12:55:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:17 smithi043 ceph-mon[41699]: osdmap e116: 8 total, 8 up, 8 in 2024-03-21T12:55:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:17 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1384663714' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81661-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:17 smithi043 ceph-mon[41699]: from='client.25726 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81661-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:17 smithi043 ceph-mon[41699]: 18.5 deep-scrub starts 2024-03-21T12:55:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:17 smithi043 ceph-mon[41699]: 18.5 deep-scrub ok 2024-03-21T12:55:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:17 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:17 smithi043 ceph-mon[41699]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:17 smithi043 ceph-mon[41699]: from='client.25726 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81661-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:17.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:17 smithi043 ceph-mon[41699]: osdmap e117: 8 total, 8 up, 8 in 2024-03-21T12:55:18.341 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:55:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:55:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:55:18.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:55:18 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:55:18.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:18 smithi155 ceph-mon[33846]: pgmap v137: 556 pgs: 104 unknown, 452 active+clean; 217 MiB data, 1015 MiB used, 714 GiB / 715 GiB avail 2024-03-21T12:55:18.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:18 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:55:18.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:18 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:55:18.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:18 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:18.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:18 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:55:18.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:18 smithi155 ceph-mon[33846]: osdmap e118: 8 total, 8 up, 8 in 2024-03-21T12:55:18.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:18 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/242152238' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-81636-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:18.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:18 smithi155 ceph-mon[33846]: from='client.25735 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-81636-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:18.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:18 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-11"}]: dispatch 2024-03-21T12:55:18.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:18 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-11"}]: dispatch 2024-03-21T12:55:18.733 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:18 smithi043 ceph-mon[41699]: pgmap v137: 556 pgs: 104 unknown, 452 active+clean; 217 MiB data, 1015 MiB used, 714 GiB / 715 GiB avail 2024-03-21T12:55:18.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:18 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:55:18.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:18 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:55:18.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:18 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:18.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:18 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:55:18.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:18 smithi043 ceph-mon[41699]: osdmap e118: 8 total, 8 up, 8 in 2024-03-21T12:55:18.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:18 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/242152238' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-81636-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:18.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:18 smithi043 ceph-mon[41699]: from='client.25735 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-81636-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:18.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:18 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-11"}]: dispatch 2024-03-21T12:55:18.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:18 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-11"}]: dispatch 2024-03-21T12:55:18.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:18 smithi043 ceph-mon[30062]: pgmap v137: 556 pgs: 104 unknown, 452 active+clean; 217 MiB data, 1015 MiB used, 714 GiB / 715 GiB avail 2024-03-21T12:55:18.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:18 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:55:18.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:18 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:55:18.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:18 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:18.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:18 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:55:18.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:18 smithi043 ceph-mon[30062]: osdmap e118: 8 total, 8 up, 8 in 2024-03-21T12:55:18.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:18 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/242152238' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-81636-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:18.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:18 smithi043 ceph-mon[30062]: from='client.25735 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-81636-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:18.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:18 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-11"}]: dispatch 2024-03-21T12:55:18.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:18 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-11"}]: dispatch 2024-03-21T12:55:19.638 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:19 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:55:19.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:19 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:55:19.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:19 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:19.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:19 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:55:19.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:19 smithi043 ceph-mon[30062]: from='client.25735 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-81636-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:19.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:19 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-11"}]': finished 2024-03-21T12:55:19.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:19 smithi043 ceph-mon[30062]: osdmap e119: 8 total, 8 up, 8 in 2024-03-21T12:55:19.639 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:19 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3955056051' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-81661-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:19.639 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:19 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:55:19.639 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:19 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:55:19.640 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:19 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:19.640 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:19 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:55:19.640 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:19 smithi043 ceph-mon[41699]: from='client.25735 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-81636-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:19.640 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:19 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-11"}]': finished 2024-03-21T12:55:19.640 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:19 smithi043 ceph-mon[41699]: osdmap e119: 8 total, 8 up, 8 in 2024-03-21T12:55:19.640 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:19 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3955056051' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-81661-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:19.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:19 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:55:19.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:19 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:55:19.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:19 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:19.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:19 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:55:19.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:19 smithi155 ceph-mon[33846]: from='client.25735 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi043-81636-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:19.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:19 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-11"}]': finished 2024-03-21T12:55:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:19 smithi155 ceph-mon[33846]: osdmap e119: 8 total, 8 up, 8 in 2024-03-21T12:55:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:19 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3955056051' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-81661-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:20.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:20 smithi155 ceph-mon[33846]: pgmap v140: 524 pgs: 32 creating+peering, 32 unknown, 460 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 258 KiB/s wr, 7 op/s 2024-03-21T12:55:20.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:20 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:20.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:20 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3955056051' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-81661-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:20.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:20 smithi155 ceph-mon[33846]: osdmap e120: 8 total, 8 up, 8 in 2024-03-21T12:55:20.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:20 smithi043 ceph-mon[30062]: pgmap v140: 524 pgs: 32 creating+peering, 32 unknown, 460 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 258 KiB/s wr, 7 op/s 2024-03-21T12:55:20.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:20 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:20.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:20 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3955056051' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-81661-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:20.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:20 smithi043 ceph-mon[30062]: osdmap e120: 8 total, 8 up, 8 in 2024-03-21T12:55:20.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:20 smithi043 ceph-mon[41699]: pgmap v140: 524 pgs: 32 creating+peering, 32 unknown, 460 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 3.0 KiB/s rd, 258 KiB/s wr, 7 op/s 2024-03-21T12:55:20.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:20 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:20.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:20 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3955056051' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi043-81661-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:20.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:20 smithi043 ceph-mon[41699]: osdmap e120: 8 total, 8 up, 8 in 2024-03-21T12:55:21.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:21 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:21.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:21 smithi155 ceph-mon[33846]: osdmap e121: 8 total, 8 up, 8 in 2024-03-21T12:55:21.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:21 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3705239809' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-81636-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:21.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:21 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:21.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:21 smithi155 ceph-mon[33846]: from='client.35771 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-81636-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:21.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:21 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:21 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:21 smithi043 ceph-mon[30062]: osdmap e121: 8 total, 8 up, 8 in 2024-03-21T12:55:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:21 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3705239809' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-81636-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:21 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:21 smithi043 ceph-mon[30062]: from='client.35771 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-81636-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:21 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:21.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:21 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:21.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:21 smithi043 ceph-mon[41699]: osdmap e121: 8 total, 8 up, 8 in 2024-03-21T12:55:21.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:21 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3705239809' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-81636-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:21.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:21 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:21.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:21 smithi043 ceph-mon[41699]: from='client.35771 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-81636-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:21.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:21 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:23.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:22 smithi155 ceph-mon[33846]: pgmap v143: 524 pgs: 96 unknown, 428 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 4 op/s 2024-03-21T12:55:23.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:22 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:23.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:22 smithi155 ceph-mon[33846]: from='client.35771 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-81636-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:23.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:22 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:23.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:22 smithi155 ceph-mon[33846]: osdmap e122: 8 total, 8 up, 8 in 2024-03-21T12:55:23.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:22 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3744184118' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-81661-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:23.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:22 smithi155 ceph-mon[33846]: from='client.25765 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-81661-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:23.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:22 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:22 smithi043 ceph-mon[30062]: pgmap v143: 524 pgs: 96 unknown, 428 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 4 op/s 2024-03-21T12:55:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:22 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:22 smithi043 ceph-mon[30062]: from='client.35771 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-81636-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:22 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:22 smithi043 ceph-mon[30062]: osdmap e122: 8 total, 8 up, 8 in 2024-03-21T12:55:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:22 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3744184118' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-81661-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:22 smithi043 ceph-mon[30062]: from='client.25765 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-81661-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:23.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:22 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:23.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:22 smithi043 ceph-mon[41699]: pgmap v143: 524 pgs: 96 unknown, 428 active+clean; 218 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 258 KiB/s wr, 4 op/s 2024-03-21T12:55:23.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:22 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:23.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:22 smithi043 ceph-mon[41699]: from='client.35771 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi043-81636-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:23.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:22 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:23.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:22 smithi043 ceph-mon[41699]: osdmap e122: 8 total, 8 up, 8 in 2024-03-21T12:55:23.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:22 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3744184118' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-81661-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:23.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:22 smithi043 ceph-mon[41699]: from='client.25765 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-81661-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:23.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:22 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:24.314 INFO:tasks.workunit.client.0.smithi043.stdout: io: Running main() from gmock_main.cc 2024-03-21T12:55:24.314 INFO:tasks.workunit.client.0.smithi043.stdout: io: [==========] Running 14 tests from 1 test suite. 2024-03-21T12:55:24.314 INFO:tasks.workunit.client.0.smithi043.stdout: io: [----------] Global test environment set-up. 2024-03-21T12:55:24.314 INFO:tasks.workunit.client.0.smithi043.stdout: io: [----------] 14 tests from NeoRadosIo 2024-03-21T12:55:24.314 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.Limits 2024-03-21T12:55:24.314 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.Limits (13964 ms) 2024-03-21T12:55:24.314 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.SimpleWrite 2024-03-21T12:55:24.314 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.SimpleWrite (4290 ms) 2024-03-21T12:55:24.315 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.ReadOp 2024-03-21T12:55:24.315 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.ReadOp (4367 ms) 2024-03-21T12:55:24.315 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.SparseRead 2024-03-21T12:55:24.315 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.SparseRead (2872 ms) 2024-03-21T12:55:24.315 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.RoundTrip 2024-03-21T12:55:24.315 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.RoundTrip (3955 ms) 2024-03-21T12:55:24.315 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.ReadIntoBuufferlist 2024-03-21T12:55:24.315 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.ReadIntoBuufferlist (2894 ms) 2024-03-21T12:55:24.315 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.OverlappingWriteRoundTrip 2024-03-21T12:55:24.315 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.OverlappingWriteRoundTrip (4190 ms) 2024-03-21T12:55:24.315 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.WriteFullRoundTrip 2024-03-21T12:55:24.315 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.WriteFullRoundTrip (3167 ms) 2024-03-21T12:55:24.315 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.AppendRoundTrip 2024-03-21T12:55:24.315 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.AppendRoundTrip (2649 ms) 2024-03-21T12:55:24.315 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.Trunc 2024-03-21T12:55:24.316 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.Trunc (3990 ms) 2024-03-21T12:55:24.316 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.Remove 2024-03-21T12:55:24.316 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.Remove (3018 ms) 2024-03-21T12:55:24.316 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.XattrsRoundTrip 2024-03-21T12:55:24.316 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.XattrsRoundTrip (3009 ms) 2024-03-21T12:55:24.316 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.RmXattr 2024-03-21T12:55:24.316 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.RmXattr (2987 ms) 2024-03-21T12:55:24.316 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ RUN ] NeoRadosIo.GetXattrs 2024-03-21T12:55:24.316 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ OK ] NeoRadosIo.GetXattrs (2993 ms) 2024-03-21T12:55:24.316 INFO:tasks.workunit.client.0.smithi043.stdout: io: [----------] 14 tests from NeoRadosIo (58346 ms total) 2024-03-21T12:55:24.316 INFO:tasks.workunit.client.0.smithi043.stdout: io: 2024-03-21T12:55:24.316 INFO:tasks.workunit.client.0.smithi043.stdout: io: [----------] Global test environment tear-down 2024-03-21T12:55:24.316 INFO:tasks.workunit.client.0.smithi043.stdout: io: [==========] 14 tests from 1 test suite ran. (58346 ms total) 2024-03-21T12:55:24.316 INFO:tasks.workunit.client.0.smithi043.stdout: io: [ PASSED ] 14 tests. 2024-03-21T12:55:24.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:24 smithi155 ceph-mon[33846]: pgmap v146: 556 pgs: 64 creating+peering, 492 active+clean; 217 MiB data, 1013 MiB used, 714 GiB / 715 GiB avail; 27 KiB/s wr, 26 op/s 2024-03-21T12:55:24.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:24 smithi155 ceph-mon[33846]: from='client.25765 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-81661-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:24.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:24 smithi155 ceph-mon[33846]: osdmap e123: 8 total, 8 up, 8 in 2024-03-21T12:55:24.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:24 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:24.733 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:24 smithi043 ceph-mon[41699]: pgmap v146: 556 pgs: 64 creating+peering, 492 active+clean; 217 MiB data, 1013 MiB used, 714 GiB / 715 GiB avail; 27 KiB/s wr, 26 op/s 2024-03-21T12:55:24.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:24 smithi043 ceph-mon[41699]: from='client.25765 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-81661-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:24.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:24 smithi043 ceph-mon[41699]: osdmap e123: 8 total, 8 up, 8 in 2024-03-21T12:55:24.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:24 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:24.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:24 smithi043 ceph-mon[30062]: pgmap v146: 556 pgs: 64 creating+peering, 492 active+clean; 217 MiB data, 1013 MiB used, 714 GiB / 715 GiB avail; 27 KiB/s wr, 26 op/s 2024-03-21T12:55:24.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:24 smithi043 ceph-mon[30062]: from='client.25765 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi043-81661-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:24.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:24 smithi043 ceph-mon[30062]: osdmap e123: 8 total, 8 up, 8 in 2024-03-21T12:55:24.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:24 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:25.319 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: Running main() from gmock_main.cc 2024-03-21T12:55:25.319 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [==========] Running 14 tests from 1 test suite. 2024-03-21T12:55:25.319 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [----------] Global test environment set-up. 2024-03-21T12:55:25.319 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps 2024-03-21T12:55:25.319 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.SetOpFlags 2024-03-21T12:55:25.319 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.SetOpFlags (11202 ms) 2024-03-21T12:55:25.320 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertExists 2024-03-21T12:55:25.320 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertExists (6972 ms) 2024-03-21T12:55:25.320 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertVersion 2024-03-21T12:55:25.320 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertVersion (5107 ms) 2024-03-21T12:55:25.320 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpXattr 2024-03-21T12:55:25.320 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpXattr (3979 ms) 2024-03-21T12:55:25.320 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.Read 2024-03-21T12:55:25.320 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.Read (4002 ms) 2024-03-21T12:55:25.320 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.Checksum 2024-03-21T12:55:25.320 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.Checksum (3131 ms) 2024-03-21T12:55:25.320 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.RWOrderedRead 2024-03-21T12:55:25.320 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.RWOrderedRead (3228 ms) 2024-03-21T12:55:25.320 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.ShortRead 2024-03-21T12:55:25.320 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.ShortRead (3638 ms) 2024-03-21T12:55:25.320 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.Exec 2024-03-21T12:55:25.321 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.Exec (3100 ms) 2024-03-21T12:55:25.321 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.Stat 2024-03-21T12:55:25.321 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.Stat (2906 ms) 2024-03-21T12:55:25.321 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.Omap 2024-03-21T12:55:25.321 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.Omap (3012 ms) 2024-03-21T12:55:25.321 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.OmapNuls 2024-03-21T12:55:25.321 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.OmapNuls (3009 ms) 2024-03-21T12:55:25.321 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.GetXattrs 2024-03-21T12:55:25.321 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.GetXattrs (2979 ms) 2024-03-21T12:55:25.321 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpExt 2024-03-21T12:55:25.321 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpExt (3011 ms) 2024-03-21T12:55:25.321 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps (59277 ms total) 2024-03-21T12:55:25.321 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: 2024-03-21T12:55:25.321 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [----------] Global test environment tear-down 2024-03-21T12:55:25.322 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [==========] 14 tests from 1 test suite ran. (59277 ms total) 2024-03-21T12:55:25.322 INFO:tasks.workunit.client.0.smithi043.stdout: read_operations: [ PASSED ] 14 tests. 2024-03-21T12:55:25.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:25 smithi155 ceph-mon[33846]: osdmap e124: 8 total, 8 up, 8 in 2024-03-21T12:55:25.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:25 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2632260724' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi043-81636-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:25.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:25 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:25.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:25 smithi043 ceph-mon[30062]: osdmap e124: 8 total, 8 up, 8 in 2024-03-21T12:55:25.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:25 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2632260724' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi043-81636-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:25.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:25 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:25.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:25 smithi043 ceph-mon[41699]: osdmap e124: 8 total, 8 up, 8 in 2024-03-21T12:55:25.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:25 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2632260724' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi043-81636-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:25.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:25 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:26.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:26 smithi155 ceph-mon[33846]: pgmap v149: 524 pgs: 64 unknown, 460 active+clean; 217 MiB data, 1013 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 27 KiB/s wr, 28 op/s 2024-03-21T12:55:26.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:26 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2632260724' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi043-81636-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:26.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:26 smithi155 ceph-mon[33846]: osdmap e125: 8 total, 8 up, 8 in 2024-03-21T12:55:26.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:26 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3434719767' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:26.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:26 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/137442980' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-81661-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:26.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:26 smithi155 ceph-mon[33846]: from='client.25693 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:26.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:26 smithi155 ceph-mon[33846]: from='client.25780 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-81661-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:26.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:26 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:26.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:26 smithi155 ceph-mon[33846]: from='client.25693 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81984-10"}]': finished 2024-03-21T12:55:26.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:26 smithi155 ceph-mon[33846]: from='client.25780 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-81661-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:26.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:26 smithi155 ceph-mon[33846]: osdmap e126: 8 total, 8 up, 8 in 2024-03-21T12:55:26.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:26 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3434719767' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:26.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:26 smithi155 ceph-mon[33846]: from='client.25693 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:26.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[30062]: pgmap v149: 524 pgs: 64 unknown, 460 active+clean; 217 MiB data, 1013 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 27 KiB/s wr, 28 op/s 2024-03-21T12:55:26.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2632260724' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi043-81636-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:26.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[30062]: osdmap e125: 8 total, 8 up, 8 in 2024-03-21T12:55:26.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3434719767' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:26.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/137442980' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-81661-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:26.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[30062]: from='client.25693 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:26.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[30062]: from='client.25780 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-81661-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:26.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:26.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[30062]: from='client.25693 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81984-10"}]': finished 2024-03-21T12:55:26.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[30062]: from='client.25780 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-81661-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:26.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[30062]: osdmap e126: 8 total, 8 up, 8 in 2024-03-21T12:55:26.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3434719767' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:26.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[30062]: from='client.25693 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:26.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[41699]: pgmap v149: 524 pgs: 64 unknown, 460 active+clean; 217 MiB data, 1013 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 27 KiB/s wr, 28 op/s 2024-03-21T12:55:26.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2632260724' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi043-81636-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:26.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[41699]: osdmap e125: 8 total, 8 up, 8 in 2024-03-21T12:55:26.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3434719767' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:26.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/137442980' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-81661-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:26.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[41699]: from='client.25693 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:26.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[41699]: from='client.25780 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-81661-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:26.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:26.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[41699]: from='client.25693 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi043-81984-10"}]': finished 2024-03-21T12:55:26.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[41699]: from='client.25780 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi043-81661-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:26.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[41699]: osdmap e126: 8 total, 8 up, 8 in 2024-03-21T12:55:26.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3434719767' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:26.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:26 smithi043 ceph-mon[41699]: from='client.25693 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81984-10"}]: dispatch 2024-03-21T12:55:27.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:27 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:55:27.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:27 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:55:27.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:27 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:27.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:27 smithi155 ceph-mon[33846]: from='client.25693 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81984-10"}]': finished 2024-03-21T12:55:27.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:27 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-13", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:55:27.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:27 smithi155 ceph-mon[33846]: osdmap e127: 8 total, 8 up, 8 in 2024-03-21T12:55:27.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:27 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-13"}]: dispatch 2024-03-21T12:55:27.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:27 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-13"}]: dispatch 2024-03-21T12:55:27.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:27 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1979719990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-81636-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:27.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:27 smithi155 ceph-mon[33846]: from='client.25786 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-81636-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:27 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:55:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:27 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:55:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:27 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:27 smithi043 ceph-mon[30062]: from='client.25693 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81984-10"}]': finished 2024-03-21T12:55:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:27 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-13", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:55:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:27 smithi043 ceph-mon[30062]: osdmap e127: 8 total, 8 up, 8 in 2024-03-21T12:55:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:27 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-13"}]: dispatch 2024-03-21T12:55:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:27 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-13"}]: dispatch 2024-03-21T12:55:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:27 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1979719990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-81636-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:27 smithi043 ceph-mon[30062]: from='client.25786 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-81636-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:27.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:27 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:55:27.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:27 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:55:27.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:27 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:27.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:27 smithi043 ceph-mon[41699]: from='client.25693 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi043-81984-10"}]': finished 2024-03-21T12:55:27.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:27 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-13", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:55:27.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:27 smithi043 ceph-mon[41699]: osdmap e127: 8 total, 8 up, 8 in 2024-03-21T12:55:27.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:27 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-13"}]: dispatch 2024-03-21T12:55:27.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:27 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-13"}]: dispatch 2024-03-21T12:55:27.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:27 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1979719990' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-81636-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:27.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:27 smithi043 ceph-mon[41699]: from='client.25786 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-81636-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:28.339 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:55:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:55:28.538 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-03-21T12:55:28.538 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] Global test environment set-up. 2024-03-21T12:55:28.538 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-03-21T12:55:28.538 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-03-21T12:55:28.538 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-03-21T12:55:28.538 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-03-21T12:55:28.538 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: 2024-03-21T12:55:28.538 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-03-21T12:55:28.538 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-03-21T12:55:28.538 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-03-21T12:55:28.538 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: 2024-03-21T12:54:25.770+0000 7fa82ecd9780 -1 failed for service _ceph-mon._tcp 2024-03-21T12:55:28.538 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: 2024-03-21T12:54:25.770+0000 7fa82ecd9780 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-03-21T12:55:28.538 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (66 ms) 2024-03-21T12:55:28.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-03-21T12:55:28.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2014 ms) 2024-03-21T12:55:28.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (2080 ms total) 2024-03-21T12:55:28.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: 2024-03-21T12:55:28.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-03-21T12:55:28.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-03-21T12:55:28.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585521d6600 2024-03-21T12:55:28.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552516200 2024-03-21T12:55:28.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552516bd0 2024-03-21T12:55:28.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251b620 2024-03-21T12:55:28.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251bc80 2024-03-21T12:55:28.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251c2a0 2024-03-21T12:55:28.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251c190 2024-03-21T12:55:28.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251c7d0 2024-03-21T12:55:28.539 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251cc80 2024-03-21T12:55:28.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251cd90 2024-03-21T12:55:28.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251d2a0 2024-03-21T12:55:28.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251d400 2024-03-21T12:55:28.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525166f0 2024-03-21T12:55:28.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251cfd0 2024-03-21T12:55:28.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251d0e0 2024-03-21T12:55:28.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251d510 2024-03-21T12:55:28.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251d620 2024-03-21T12:55:28.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251dbe0 2024-03-21T12:55:28.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251d730 2024-03-21T12:55:28.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251dcf0 2024-03-21T12:55:28.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251d880 2024-03-21T12:55:28.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251da00 2024-03-21T12:55:28.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251de00 2024-03-21T12:55:28.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251df10 2024-03-21T12:55:28.540 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251e4d0 2024-03-21T12:55:28.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251e020 2024-03-21T12:55:28.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251c8e0 2024-03-21T12:55:28.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251c9f0 2024-03-21T12:55:28.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251cb00 2024-03-21T12:55:28.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251ea90 2024-03-21T12:55:28.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251e5e0 2024-03-21T12:55:28.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251e730 2024-03-21T12:55:28.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251e840 2024-03-21T12:55:28.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251eba0 2024-03-21T12:55:28.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251f050 2024-03-21T12:55:28.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251ecb0 2024-03-21T12:55:28.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251edc0 2024-03-21T12:55:28.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251eed0 2024-03-21T12:55:28.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251e950 2024-03-21T12:55:28.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251f1d0 2024-03-21T12:55:28.541 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251f680 2024-03-21T12:55:28.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251f7b0 2024-03-21T12:55:28.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251fd70 2024-03-21T12:55:28.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251f2e0 2024-03-21T12:55:28.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251f410 2024-03-21T12:55:28.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251f520 2024-03-21T12:55:28.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251fe80 2024-03-21T12:55:28.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251f8c0 2024-03-21T12:55:28.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251f9f0 2024-03-21T12:55:28.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251fb00 2024-03-21T12:55:28.542 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251fc30 2024-03-21T12:55:28.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:55:28 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:55:28.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:28 smithi155 ceph-mon[33846]: pgmap v152: 484 pgs: 32 unknown, 452 active+clean; 217 MiB data, 1013 MiB used, 714 GiB / 715 GiB avail 2024-03-21T12:55:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:28 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3844963195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:28 smithi155 ceph-mon[33846]: from='client.25798 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3844963195' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:28 smithi155 ceph-mon[33846]: from='client.25798 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3844963195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:55:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:28 smithi155 ceph-mon[33846]: from='client.25798 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:55:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:28 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-13"}]': finished 2024-03-21T12:55:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:28 smithi155 ceph-mon[33846]: from='client.25786 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-81636-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:28 smithi155 ceph-mon[33846]: from='client.25798 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:55:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3844963195' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-81984-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:28 smithi155 ceph-mon[33846]: osdmap e128: 8 total, 8 up, 8 in 2024-03-21T12:55:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-13", "mode": "writeback"}]: dispatch 2024-03-21T12:55:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:28 smithi155 ceph-mon[33846]: from='client.25798 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-81984-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:28.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:28 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-13", "mode": "writeback"}]: dispatch 2024-03-21T12:55:28.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1821815196' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-81661-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:28.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:28 smithi155 ceph-mon[33846]: from='client.25795 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-81661-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:28.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[30062]: pgmap v152: 484 pgs: 32 unknown, 452 active+clean; 217 MiB data, 1013 MiB used, 714 GiB / 715 GiB avail 2024-03-21T12:55:28.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:28.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3844963195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:28.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[30062]: from='client.25798 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:28.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3844963195' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:28.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[30062]: from='client.25798 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:28.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3844963195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:55:28.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[30062]: from='client.25798 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:55:28.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:28.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-13"}]': finished 2024-03-21T12:55:28.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[30062]: from='client.25786 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-81636-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:28.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[30062]: from='client.25798 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:55:28.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3844963195' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-81984-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:28.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[30062]: osdmap e128: 8 total, 8 up, 8 in 2024-03-21T12:55:28.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-13", "mode": "writeback"}]: dispatch 2024-03-21T12:55:28.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[30062]: from='client.25798 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-81984-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:28.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-13", "mode": "writeback"}]: dispatch 2024-03-21T12:55:28.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1821815196' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-81661-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:28.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[30062]: from='client.25795 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-81661-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:28.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[41699]: pgmap v152: 484 pgs: 32 unknown, 452 active+clean; 217 MiB data, 1013 MiB used, 714 GiB / 715 GiB avail 2024-03-21T12:55:28.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:28.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3844963195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:28.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[41699]: from='client.25798 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:28.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3844963195' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:28.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[41699]: from='client.25798 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:28.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3844963195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:55:28.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[41699]: from='client.25798 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:55:28.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:28.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-13"}]': finished 2024-03-21T12:55:28.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[41699]: from='client.25786 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi043-81636-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:28.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[41699]: from='client.25798 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:55:28.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3844963195' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-81984-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:28.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[41699]: osdmap e128: 8 total, 8 up, 8 in 2024-03-21T12:55:28.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-13", "mode": "writeback"}]: dispatch 2024-03-21T12:55:28.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[41699]: from='client.25798 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-81984-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:28.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-13", "mode": "writeback"}]: dispatch 2024-03-21T12:55:28.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1821815196' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-81661-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:28.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:28 smithi043 ceph-mon[41699]: from='client.25795 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-81661-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:29 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:55:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:29 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:29 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:55:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:29 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-13", "mode": "writeback"}]': finished 2024-03-21T12:55:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:29 smithi155 ceph-mon[33846]: from='client.25795 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-81661-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:29.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:29 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:55:29.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:29 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:29.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:29 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:55:29.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:29 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-13", "mode": "writeback"}]': finished 2024-03-21T12:55:29.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:29 smithi043 ceph-mon[41699]: from='client.25795 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-81661-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:29 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:55:29.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:29 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:29.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:29 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:55:29.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:29 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-13", "mode": "writeback"}]': finished 2024-03-21T12:55:29.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:29 smithi043 ceph-mon[30062]: from='client.25795 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi043-81661-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:30.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:30 smithi155 ceph-mon[33846]: pgmap v155: 516 pgs: 32 unknown, 32 creating+peering, 452 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 99 KiB/s wr, 99 op/s 2024-03-21T12:55:30.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:30 smithi155 ceph-mon[33846]: osdmap e129: 8 total, 8 up, 8 in 2024-03-21T12:55:30.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:30 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:30.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:30 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.0"}]: dispatch 2024-03-21T12:55:30.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:30 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.1"}]: dispatch 2024-03-21T12:55:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:30 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.2"}]: dispatch 2024-03-21T12:55:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:30 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.3"}]: dispatch 2024-03-21T12:55:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:30 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.4"}]: dispatch 2024-03-21T12:55:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:30 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.5"}]: dispatch 2024-03-21T12:55:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:30 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.6"}]: dispatch 2024-03-21T12:55:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:30 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.7"}]: dispatch 2024-03-21T12:55:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:30 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.8"}]: dispatch 2024-03-21T12:55:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:30 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.9"}]: dispatch 2024-03-21T12:55:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:30 smithi155 ceph-mon[33846]: 161.0 scrub starts 2024-03-21T12:55:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:30 smithi155 ceph-mon[33846]: 161.0 scrub ok 2024-03-21T12:55:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:30 smithi155 ceph-mon[33846]: from='client.25798 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-81984-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]': finished 2024-03-21T12:55:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:30 smithi155 ceph-mon[33846]: osdmap e130: 8 total, 8 up, 8 in 2024-03-21T12:55:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:30 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4044715384' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-81636-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:30.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:30 smithi155 ceph-mon[33846]: from='client.35813 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-81636-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:30.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[30062]: pgmap v155: 516 pgs: 32 unknown, 32 creating+peering, 452 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 99 KiB/s wr, 99 op/s 2024-03-21T12:55:30.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[30062]: osdmap e129: 8 total, 8 up, 8 in 2024-03-21T12:55:30.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:30.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.0"}]: dispatch 2024-03-21T12:55:30.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.1"}]: dispatch 2024-03-21T12:55:30.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.2"}]: dispatch 2024-03-21T12:55:30.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.3"}]: dispatch 2024-03-21T12:55:30.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.4"}]: dispatch 2024-03-21T12:55:30.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.5"}]: dispatch 2024-03-21T12:55:30.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.6"}]: dispatch 2024-03-21T12:55:30.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.7"}]: dispatch 2024-03-21T12:55:30.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.8"}]: dispatch 2024-03-21T12:55:30.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.9"}]: dispatch 2024-03-21T12:55:30.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[30062]: 161.0 scrub starts 2024-03-21T12:55:30.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[30062]: 161.0 scrub ok 2024-03-21T12:55:30.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[30062]: from='client.25798 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-81984-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]': finished 2024-03-21T12:55:30.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[30062]: osdmap e130: 8 total, 8 up, 8 in 2024-03-21T12:55:30.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4044715384' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-81636-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:30.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[30062]: from='client.35813 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-81636-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:30.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[41699]: pgmap v155: 516 pgs: 32 unknown, 32 creating+peering, 452 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 99 KiB/s wr, 99 op/s 2024-03-21T12:55:30.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[41699]: osdmap e129: 8 total, 8 up, 8 in 2024-03-21T12:55:30.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:30.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.0"}]: dispatch 2024-03-21T12:55:30.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.1"}]: dispatch 2024-03-21T12:55:30.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.2"}]: dispatch 2024-03-21T12:55:30.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.3"}]: dispatch 2024-03-21T12:55:30.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.4"}]: dispatch 2024-03-21T12:55:30.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.5"}]: dispatch 2024-03-21T12:55:30.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.6"}]: dispatch 2024-03-21T12:55:30.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.7"}]: dispatch 2024-03-21T12:55:30.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.8"}]: dispatch 2024-03-21T12:55:30.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "161.9"}]: dispatch 2024-03-21T12:55:30.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[41699]: 161.0 scrub starts 2024-03-21T12:55:30.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[41699]: 161.0 scrub ok 2024-03-21T12:55:30.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[41699]: from='client.25798 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi043-81984-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]': finished 2024-03-21T12:55:30.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[41699]: osdmap e130: 8 total, 8 up, 8 in 2024-03-21T12:55:30.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4044715384' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-81636-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:30.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:30 smithi043 ceph-mon[41699]: from='client.35813 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-81636-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:31.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.0"}]: dispatch 2024-03-21T12:55:31.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.1"}]: dispatch 2024-03-21T12:55:31.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.2"}]: dispatch 2024-03-21T12:55:31.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.3"}]: dispatch 2024-03-21T12:55:31.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.4"}]: dispatch 2024-03-21T12:55:31.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.5"}]: dispatch 2024-03-21T12:55:31.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.6"}]: dispatch 2024-03-21T12:55:31.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.7"}]: dispatch 2024-03-21T12:55:31.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.8"}]: dispatch 2024-03-21T12:55:31.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.9"}]: dispatch 2024-03-21T12:55:31.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: 161.8 scrub starts 2024-03-21T12:55:31.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: 161.8 scrub ok 2024-03-21T12:55:31.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: 161.7 scrub starts 2024-03-21T12:55:31.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: 161.7 scrub ok 2024-03-21T12:55:31.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: 161.5 scrub starts 2024-03-21T12:55:31.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: 161.5 scrub ok 2024-03-21T12:55:31.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: 161.9 deep-scrub starts 2024-03-21T12:55:31.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: 161.9 deep-scrub ok 2024-03-21T12:55:31.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:31.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: 161.1 scrub starts 2024-03-21T12:55:31.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: 161.1 scrub ok 2024-03-21T12:55:31.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: 161.3 scrub starts 2024-03-21T12:55:31.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: 161.3 scrub ok 2024-03-21T12:55:31.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: 161.2 deep-scrub starts 2024-03-21T12:55:31.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: 161.2 deep-scrub ok 2024-03-21T12:55:31.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: from='client.35813 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-81636-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:31.960 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: osdmap e131: 8 total, 8 up, 8 in 2024-03-21T12:55:31.960 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1849437761' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-81661-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:31.960 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:31 smithi155 ceph-mon[33846]: from='client.25810 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-81661-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:31.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.0"}]: dispatch 2024-03-21T12:55:31.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.1"}]: dispatch 2024-03-21T12:55:31.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.2"}]: dispatch 2024-03-21T12:55:31.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.3"}]: dispatch 2024-03-21T12:55:31.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.4"}]: dispatch 2024-03-21T12:55:31.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.5"}]: dispatch 2024-03-21T12:55:31.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.6"}]: dispatch 2024-03-21T12:55:31.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.7"}]: dispatch 2024-03-21T12:55:31.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.8"}]: dispatch 2024-03-21T12:55:31.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.9"}]: dispatch 2024-03-21T12:55:31.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: 161.8 scrub starts 2024-03-21T12:55:31.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: 161.8 scrub ok 2024-03-21T12:55:31.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: 161.7 scrub starts 2024-03-21T12:55:31.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: 161.7 scrub ok 2024-03-21T12:55:31.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: 161.5 scrub starts 2024-03-21T12:55:31.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: 161.5 scrub ok 2024-03-21T12:55:31.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: 161.9 deep-scrub starts 2024-03-21T12:55:31.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: 161.9 deep-scrub ok 2024-03-21T12:55:31.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:31.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: 161.1 scrub starts 2024-03-21T12:55:31.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: 161.1 scrub ok 2024-03-21T12:55:31.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: 161.3 scrub starts 2024-03-21T12:55:31.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: 161.3 scrub ok 2024-03-21T12:55:31.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: 161.2 deep-scrub starts 2024-03-21T12:55:31.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: 161.2 deep-scrub ok 2024-03-21T12:55:31.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: from='client.35813 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-81636-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:31.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: osdmap e131: 8 total, 8 up, 8 in 2024-03-21T12:55:31.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1849437761' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-81661-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:31.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[30062]: from='client.25810 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-81661-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:31.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.0"}]: dispatch 2024-03-21T12:55:31.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.1"}]: dispatch 2024-03-21T12:55:31.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.2"}]: dispatch 2024-03-21T12:55:31.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.3"}]: dispatch 2024-03-21T12:55:31.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.4"}]: dispatch 2024-03-21T12:55:31.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.5"}]: dispatch 2024-03-21T12:55:31.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.6"}]: dispatch 2024-03-21T12:55:31.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.7"}]: dispatch 2024-03-21T12:55:31.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.8"}]: dispatch 2024-03-21T12:55:31.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "161.9"}]: dispatch 2024-03-21T12:55:31.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: 161.8 scrub starts 2024-03-21T12:55:31.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: 161.8 scrub ok 2024-03-21T12:55:31.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: 161.7 scrub starts 2024-03-21T12:55:31.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: 161.7 scrub ok 2024-03-21T12:55:31.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: 161.5 scrub starts 2024-03-21T12:55:31.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: 161.5 scrub ok 2024-03-21T12:55:31.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: 161.9 deep-scrub starts 2024-03-21T12:55:31.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: 161.9 deep-scrub ok 2024-03-21T12:55:31.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:31.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: 161.1 scrub starts 2024-03-21T12:55:31.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: 161.1 scrub ok 2024-03-21T12:55:31.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: 161.3 scrub starts 2024-03-21T12:55:31.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: 161.3 scrub ok 2024-03-21T12:55:31.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: 161.2 deep-scrub starts 2024-03-21T12:55:31.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: 161.2 deep-scrub ok 2024-03-21T12:55:31.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: from='client.35813 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi043-81636-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:31.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: osdmap e131: 8 total, 8 up, 8 in 2024-03-21T12:55:31.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1849437761' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-81661-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:31.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:31 smithi043 ceph-mon[41699]: from='client.25810 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-81661-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:32.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:32 smithi155 ceph-mon[33846]: 161.6 scrub starts 2024-03-21T12:55:32.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:32 smithi155 ceph-mon[33846]: 161.6 scrub ok 2024-03-21T12:55:32.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:32 smithi155 ceph-mon[33846]: pgmap v158: 492 pgs: 40 unknown, 452 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 100 KiB/s wr, 101 op/s 2024-03-21T12:55:32.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:32 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:32.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:32 smithi155 ceph-mon[33846]: from='client.25810 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-81661-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:32.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:32 smithi155 ceph-mon[33846]: osdmap e132: 8 total, 8 up, 8 in 2024-03-21T12:55:32.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:32 smithi043 ceph-mon[30062]: 161.6 scrub starts 2024-03-21T12:55:32.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:32 smithi043 ceph-mon[30062]: 161.6 scrub ok 2024-03-21T12:55:32.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:32 smithi043 ceph-mon[30062]: pgmap v158: 492 pgs: 40 unknown, 452 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 100 KiB/s wr, 101 op/s 2024-03-21T12:55:32.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:32 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:32.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:32 smithi043 ceph-mon[30062]: from='client.25810 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-81661-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:32.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:32 smithi043 ceph-mon[30062]: osdmap e132: 8 total, 8 up, 8 in 2024-03-21T12:55:32.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:32 smithi043 ceph-mon[41699]: 161.6 scrub starts 2024-03-21T12:55:32.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:32 smithi043 ceph-mon[41699]: 161.6 scrub ok 2024-03-21T12:55:32.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:32 smithi043 ceph-mon[41699]: pgmap v158: 492 pgs: 40 unknown, 452 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 100 KiB/s wr, 101 op/s 2024-03-21T12:55:32.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:32 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:32.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:32 smithi043 ceph-mon[41699]: from='client.25810 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi043-81661-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:32.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:32 smithi043 ceph-mon[41699]: osdmap e132: 8 total, 8 up, 8 in 2024-03-21T12:55:33.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:33 smithi155 ceph-mon[33846]: 161.4 scrub starts 2024-03-21T12:55:33.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:33 smithi155 ceph-mon[33846]: 161.4 scrub ok 2024-03-21T12:55:33.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:33 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:33.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:33 smithi155 ceph-mon[33846]: osdmap e133: 8 total, 8 up, 8 in 2024-03-21T12:55:33.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:33 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3842112853' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-81636-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:33.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:33 smithi155 ceph-mon[33846]: from='client.25816 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-81636-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:33.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:33 smithi043 ceph-mon[30062]: 161.4 scrub starts 2024-03-21T12:55:33.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:33 smithi043 ceph-mon[30062]: 161.4 scrub ok 2024-03-21T12:55:33.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:33 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:33.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:33 smithi043 ceph-mon[30062]: osdmap e133: 8 total, 8 up, 8 in 2024-03-21T12:55:33.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:33 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3842112853' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-81636-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:33.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:33 smithi043 ceph-mon[30062]: from='client.25816 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-81636-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:33.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:33 smithi043 ceph-mon[41699]: 161.4 scrub starts 2024-03-21T12:55:33.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:33 smithi043 ceph-mon[41699]: 161.4 scrub ok 2024-03-21T12:55:33.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:33 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:33.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:33 smithi043 ceph-mon[41699]: osdmap e133: 8 total, 8 up, 8 in 2024-03-21T12:55:33.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:33 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3842112853' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-81636-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:33.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:33 smithi043 ceph-mon[41699]: from='client.25816 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-81636-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:34.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:34 smithi155 ceph-mon[33846]: pgmap v161: 492 pgs: 492 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 4.2 KiB/s wr, 77 op/s 2024-03-21T12:55:35.032 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:34 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:35.032 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:34 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:55:35.032 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:34 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:35.032 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:34 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pg_num","val":"11"}]: dispatch 2024-03-21T12:55:35.032 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:34 smithi155 ceph-mon[33846]: from='client.24697 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pg_num","val":"11"}]: dispatch 2024-03-21T12:55:35.032 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:34 smithi155 ceph-mon[33846]: from='client.25816 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-81636-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:35.032 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:34 smithi155 ceph-mon[33846]: from='client.24697 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pg_num","val":"11"}]': finished 2024-03-21T12:55:35.032 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:34 smithi155 ceph-mon[33846]: osdmap e134: 8 total, 8 up, 8 in 2024-03-21T12:55:35.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:34 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:35.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:34 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:35.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:34 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-21T12:55:35.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:34 smithi155 ceph-mon[33846]: from='client.24697 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-21T12:55:35.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:34 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2679150393' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-81661-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:35.033 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:34 smithi155 ceph-mon[33846]: from='client.25822 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-81661-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:35.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[30062]: pgmap v161: 492 pgs: 492 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 4.2 KiB/s wr, 77 op/s 2024-03-21T12:55:35.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:35.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:55:35.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:35.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pg_num","val":"11"}]: dispatch 2024-03-21T12:55:35.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[30062]: from='client.24697 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pg_num","val":"11"}]: dispatch 2024-03-21T12:55:35.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[30062]: from='client.25816 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-81636-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:35.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[30062]: from='client.24697 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pg_num","val":"11"}]': finished 2024-03-21T12:55:35.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[30062]: osdmap e134: 8 total, 8 up, 8 in 2024-03-21T12:55:35.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:35.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:35.034 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-21T12:55:35.035 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[30062]: from='client.24697 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-21T12:55:35.035 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2679150393' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-81661-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:35.035 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[30062]: from='client.25822 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-81661-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:35.035 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[41699]: pgmap v161: 492 pgs: 492 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 4.2 KiB/s wr, 77 op/s 2024-03-21T12:55:35.035 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:35.035 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:55:35.035 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:35.035 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pg_num","val":"11"}]: dispatch 2024-03-21T12:55:35.035 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[41699]: from='client.24697 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pg_num","val":"11"}]: dispatch 2024-03-21T12:55:35.035 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[41699]: from='client.25816 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi043-81636-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:35.035 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[41699]: from='client.24697 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pg_num","val":"11"}]': finished 2024-03-21T12:55:35.035 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[41699]: osdmap e134: 8 total, 8 up, 8 in 2024-03-21T12:55:35.036 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:35.036 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:35.036 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-21T12:55:35.036 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[41699]: from='client.24697 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-21T12:55:35.036 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2679150393' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-81661-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:35.036 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:34 smithi043 ceph-mon[41699]: from='client.25822 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-81661-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:36.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:36 smithi155 ceph-mon[33846]: pgmap v164: 524 pgs: 64 unknown, 460 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 4.0 KiB/s wr, 77 op/s 2024-03-21T12:55:36.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:36 smithi155 ceph-mon[33846]: from='client.24697 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pgp_num","val":"11"}]': finished 2024-03-21T12:55:36.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:36 smithi155 ceph-mon[33846]: from='client.25822 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-81661-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:36.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:36 smithi155 ceph-mon[33846]: osdmap e135: 8 total, 8 up, 8 in 2024-03-21T12:55:36.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:36.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:36.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pg_num","val":"11"}]: dispatch 2024-03-21T12:55:36.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:36 smithi155 ceph-mon[33846]: from='client.24697 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pg_num","val":"11"}]: dispatch 2024-03-21T12:55:36.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:36 smithi043 ceph-mon[30062]: pgmap v164: 524 pgs: 64 unknown, 460 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 4.0 KiB/s wr, 77 op/s 2024-03-21T12:55:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:36 smithi043 ceph-mon[30062]: from='client.24697 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pgp_num","val":"11"}]': finished 2024-03-21T12:55:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:36 smithi043 ceph-mon[30062]: from='client.25822 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-81661-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:36 smithi043 ceph-mon[30062]: osdmap e135: 8 total, 8 up, 8 in 2024-03-21T12:55:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pg_num","val":"11"}]: dispatch 2024-03-21T12:55:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:36 smithi043 ceph-mon[30062]: from='client.24697 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pg_num","val":"11"}]: dispatch 2024-03-21T12:55:36.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:36 smithi043 ceph-mon[41699]: pgmap v164: 524 pgs: 64 unknown, 460 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 59 KiB/s rd, 4.0 KiB/s wr, 77 op/s 2024-03-21T12:55:36.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:36 smithi043 ceph-mon[41699]: from='client.24697 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pgp_num","val":"11"}]': finished 2024-03-21T12:55:36.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:36 smithi043 ceph-mon[41699]: from='client.25822 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi043-81661-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:36.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:36 smithi043 ceph-mon[41699]: osdmap e135: 8 total, 8 up, 8 in 2024-03-21T12:55:36.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:36.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:36.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pg_num","val":"11"}]: dispatch 2024-03-21T12:55:36.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:36 smithi043 ceph-mon[41699]: from='client.24697 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pg_num","val":"11"}]: dispatch 2024-03-21T12:55:37.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:37 smithi155 ceph-mon[33846]: from='client.24697 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pg_num","val":"11"}]': finished 2024-03-21T12:55:37.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:37 smithi155 ceph-mon[33846]: osdmap e136: 8 total, 8 up, 8 in 2024-03-21T12:55:37.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:37 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:37.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:37 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3842256291' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-81636-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:37.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:37 smithi155 ceph-mon[33846]: from='client.25828 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-81636-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:37.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:37 smithi043 ceph-mon[30062]: from='client.24697 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pg_num","val":"11"}]': finished 2024-03-21T12:55:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:37 smithi043 ceph-mon[30062]: osdmap e136: 8 total, 8 up, 8 in 2024-03-21T12:55:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:37 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:37 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3842256291' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-81636-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:37 smithi043 ceph-mon[30062]: from='client.25828 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-81636-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:37.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:37 smithi043 ceph-mon[41699]: from='client.24697 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pg_num","val":"11"}]': finished 2024-03-21T12:55:37.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:37 smithi043 ceph-mon[41699]: osdmap e136: 8 total, 8 up, 8 in 2024-03-21T12:55:37.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:37 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:37.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:37 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3842256291' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-81636-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:37.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:37 smithi043 ceph-mon[41699]: from='client.25828 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-81636-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:38.333 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:55:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:55:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:55:38.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:38 smithi155 ceph-mon[33846]: pgmap v167: 492 pgs: 32 unknown, 460 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-03-21T12:55:38.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:38 smithi155 ceph-mon[33846]: from='client.25828 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-81636-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:38.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:38 smithi155 ceph-mon[33846]: osdmap e137: 8 total, 8 up, 8 in 2024-03-21T12:55:38.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:38 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:38.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:38 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1959088402' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-81661-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:38.708 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:55:38 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:55:38.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:38 smithi043 ceph-mon[30062]: pgmap v167: 492 pgs: 32 unknown, 460 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-03-21T12:55:38.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:38 smithi043 ceph-mon[30062]: from='client.25828 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-81636-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:38.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:38 smithi043 ceph-mon[30062]: osdmap e137: 8 total, 8 up, 8 in 2024-03-21T12:55:38.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:38 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:38.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:38 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1959088402' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-81661-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:38.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:38 smithi043 ceph-mon[41699]: pgmap v167: 492 pgs: 32 unknown, 460 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-03-21T12:55:38.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:38 smithi043 ceph-mon[41699]: from='client.25828 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi043-81636-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:38.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:38 smithi043 ceph-mon[41699]: osdmap e137: 8 total, 8 up, 8 in 2024-03-21T12:55:38.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:38 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:38.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:38 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1959088402' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-81661-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:39.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:39 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:55:39.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:39 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1959088402' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-81661-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:39.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:39 smithi155 ceph-mon[33846]: osdmap e138: 8 total, 8 up, 8 in 2024-03-21T12:55:39.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:39 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:39.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:39 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-21T12:55:39.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:39 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:39.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:39 smithi155 ceph-mon[33846]: from='client.24709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:39.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:39 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:55:39.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:39 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1959088402' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-81661-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:39.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:39 smithi043 ceph-mon[30062]: osdmap e138: 8 total, 8 up, 8 in 2024-03-21T12:55:39.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:39 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:39.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:39 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-21T12:55:39.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:39 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:39.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:39 smithi043 ceph-mon[30062]: from='client.24709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:39.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:39 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:55:39.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:39 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1959088402' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi043-81661-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:39.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:39 smithi043 ceph-mon[41699]: osdmap e138: 8 total, 8 up, 8 in 2024-03-21T12:55:39.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:39 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:39.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:39 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-21T12:55:39.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:39 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:39.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:39 smithi043 ceph-mon[41699]: from='client.24709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:40.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:40 smithi155 ceph-mon[33846]: pgmap v170: 492 pgs: 32 creating+peering, 460 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 8.0 KiB/s wr, 10 op/s 2024-03-21T12:55:40.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:40 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:40.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:40 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:40.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:40 smithi155 ceph-mon[33846]: from='client.24709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:40.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:40 smithi155 ceph-mon[33846]: osdmap e139: 8 total, 8 up, 8 in 2024-03-21T12:55:40.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:40 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3844963195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:40.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:40 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2984299703' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81636-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:40.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:40 smithi155 ceph-mon[33846]: from='client.25798 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:40.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:40 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "app2"}]: dispatch 2024-03-21T12:55:40.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:40 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:40.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:40 smithi155 ceph-mon[33846]: from='client.24709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:40.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:40 smithi043 ceph-mon[30062]: pgmap v170: 492 pgs: 32 creating+peering, 460 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 8.0 KiB/s wr, 10 op/s 2024-03-21T12:55:40.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:40 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:40.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:40 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:40.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:40 smithi043 ceph-mon[30062]: from='client.24709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:40.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:40 smithi043 ceph-mon[30062]: osdmap e139: 8 total, 8 up, 8 in 2024-03-21T12:55:40.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:40 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3844963195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:40.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:40 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2984299703' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81636-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:40.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:40 smithi043 ceph-mon[30062]: from='client.25798 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:40.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:40 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "app2"}]: dispatch 2024-03-21T12:55:40.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:40 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:40.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:40 smithi043 ceph-mon[30062]: from='client.24709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:40.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:40 smithi043 ceph-mon[41699]: pgmap v170: 492 pgs: 32 creating+peering, 460 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 3.2 KiB/s rd, 8.0 KiB/s wr, 10 op/s 2024-03-21T12:55:40.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:40 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:40.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:40 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:40.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:40 smithi043 ceph-mon[41699]: from='client.24709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:40.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:40 smithi043 ceph-mon[41699]: osdmap e139: 8 total, 8 up, 8 in 2024-03-21T12:55:40.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:40 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3844963195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:40.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:40 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2984299703' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81636-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:40.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:40 smithi043 ceph-mon[41699]: from='client.25798 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:40.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:40 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "app2"}]: dispatch 2024-03-21T12:55:40.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:40 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:40.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:40 smithi043 ceph-mon[41699]: from='client.24709 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:41.357 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started api_snapshots: Running main() from gmock_main.cc 2024-03-21T12:55:41.357 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [==========] Running 13 tests from 4 test suites. 2024-03-21T12:55:41.357 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] Global test environment set-up. 2024-03-21T12:55:41.357 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-03-21T12:55:41.357 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-03-21T12:55:41.357 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (5170 ms) 2024-03-21T12:55:41.357 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-03-21T12:55:41.357 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (4525 ms) 2024-03-21T12:55:41.357 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-03-21T12:55:41.357 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (3755 ms) 2024-03-21T12:55:41.357 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-03-21T12:55:41.358 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (2116 ms) 2024-03-21T12:55:41.358 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (15566 ms total) 2024-03-21T12:55:41.358 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: 2024-03-21T12:55:41.358 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-03-21T12:55:41.358 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-03-21T12:55:41.358 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (5101 ms) 2024-03-21T12:55:41.358 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-03-21T12:55:41.358 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (5245 ms) 2024-03-21T12:55:41.358 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-03-21T12:55:41.358 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (6747 ms) 2024-03-21T12:55:41.358 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged (17093 ms total) 2024-03-21T12:55:41.358 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: 2024-03-21T12:55:41.358 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-03-21T12:55:41.358 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-03-21T12:55:41.358 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (2993 ms) 2024-03-21T12:55:41.359 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-03-21T12:55:41.359 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (2008 ms) 2024-03-21T12:55:41.359 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-03-21T12:55:41.359 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (1997 ms) 2024-03-21T12:55:41.359 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-03-21T12:55:41.359 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (1997 ms) 2024-03-21T12:55:41.359 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (8995 ms total) 2024-03-21T12:55:41.359 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: 2024-03-21T12:55:41.359 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-03-21T12:55:41.359 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-03-21T12:55:41.359 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (4025 ms) 2024-03-21T12:55:41.359 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-03-21T12:55:41.359 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (4031 ms) 2024-03-21T12:55:41.359 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (8056 ms total) 2024-03-21T12:55:41.359 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: 2024-03-21T12:55:41.359 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [----------] Global test environment tear-down 2024-03-21T12:55:41.360 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [==========] 13 tests from 4 test suites ran. (75637 ms total) 2024-03-21T12:55:41.360 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots: [ PASSED ] 13 tests. 2024-03-21T12:55:41.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:41.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2984299703' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81636-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:41.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:41 smithi155 ceph-mon[33846]: from='client.25798 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]': finished 2024-03-21T12:55:41.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:41 smithi155 ceph-mon[33846]: from='client.24709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:41.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:41 smithi155 ceph-mon[33846]: osdmap e140: 8 total, 8 up, 8 in 2024-03-21T12:55:41.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3844963195' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:41.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81791-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-03-21T12:55:41.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-21T12:55:41.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:41 smithi155 ceph-mon[33846]: from='client.25798 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:41.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:41 smithi155 ceph-mon[33846]: from='client.24709 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-21T12:55:41.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/348603124' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81661-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:41.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:41 smithi155 ceph-mon[33846]: from='client.25843 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81661-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:41.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:41.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2984299703' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81636-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:41.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:41 smithi043 ceph-mon[30062]: from='client.25798 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]': finished 2024-03-21T12:55:41.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:41 smithi043 ceph-mon[30062]: from='client.24709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:41.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:41 smithi043 ceph-mon[30062]: osdmap e140: 8 total, 8 up, 8 in 2024-03-21T12:55:41.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3844963195' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:41.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81791-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-03-21T12:55:41.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-21T12:55:41.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:41 smithi043 ceph-mon[30062]: from='client.25798 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:41.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:41 smithi043 ceph-mon[30062]: from='client.24709 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-21T12:55:41.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/348603124' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81661-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:41.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:41 smithi043 ceph-mon[30062]: from='client.25843 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81661-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:41.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:41.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2984299703' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81636-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:41.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:41 smithi043 ceph-mon[41699]: from='client.25798 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]': finished 2024-03-21T12:55:41.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:41 smithi043 ceph-mon[41699]: from='client.24709 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi043-81791-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:41.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:41 smithi043 ceph-mon[41699]: osdmap e140: 8 total, 8 up, 8 in 2024-03-21T12:55:41.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3844963195' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:41.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81791-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-03-21T12:55:41.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-21T12:55:41.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:41 smithi043 ceph-mon[41699]: from='client.25798 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]: dispatch 2024-03-21T12:55:41.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:41 smithi043 ceph-mon[41699]: from='client.24709 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-21T12:55:41.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/348603124' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81661-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:41.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:41 smithi043 ceph-mon[41699]: from='client.25843 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81661-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:42.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:42 smithi155 ceph-mon[33846]: pgmap v173: 484 pgs: 64 unknown, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-03-21T12:55:42.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:42.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:42 smithi155 ceph-mon[33846]: from='client.25798 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]': finished 2024-03-21T12:55:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:42 smithi155 ceph-mon[33846]: from='client.24709 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-21T12:55:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:42 smithi155 ceph-mon[33846]: from='client.25843 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81661-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:42 smithi155 ceph-mon[33846]: osdmap e141: 8 total, 8 up, 8 in 2024-03-21T12:55:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-21T12:55:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2656335356' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82022-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:42 smithi155 ceph-mon[33846]: from='client.24709 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-21T12:55:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:42 smithi155 ceph-mon[33846]: from='client.35861 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82022-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:42.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:42 smithi043 ceph-mon[30062]: pgmap v173: 484 pgs: 64 unknown, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-03-21T12:55:42.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:42.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:42 smithi043 ceph-mon[30062]: from='client.25798 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]': finished 2024-03-21T12:55:42.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:42 smithi043 ceph-mon[30062]: from='client.24709 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-21T12:55:42.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:42 smithi043 ceph-mon[30062]: from='client.25843 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81661-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:42.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:42 smithi043 ceph-mon[30062]: osdmap e141: 8 total, 8 up, 8 in 2024-03-21T12:55:42.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-21T12:55:42.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2656335356' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82022-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:42.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:42 smithi043 ceph-mon[30062]: from='client.24709 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-21T12:55:42.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:42 smithi043 ceph-mon[30062]: from='client.35861 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82022-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:42.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:42.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:42 smithi043 ceph-mon[41699]: pgmap v173: 484 pgs: 64 unknown, 420 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-03-21T12:55:42.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:42.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:42 smithi043 ceph-mon[41699]: from='client.25798 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi043-81984-15"}]': finished 2024-03-21T12:55:42.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:42 smithi043 ceph-mon[41699]: from='client.24709 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-21T12:55:42.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:42 smithi043 ceph-mon[41699]: from='client.25843 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81661-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:42.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:42 smithi043 ceph-mon[41699]: osdmap e141: 8 total, 8 up, 8 in 2024-03-21T12:55:42.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-21T12:55:42.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2656335356' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82022-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:42.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:42 smithi043 ceph-mon[41699]: from='client.24709 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-21T12:55:42.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:42 smithi043 ceph-mon[41699]: from='client.35861 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82022-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:42.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:43 smithi155 ceph-mon[33846]: from='client.24709 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-21T12:55:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:43 smithi155 ceph-mon[33846]: from='client.35861 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82022-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:43 smithi155 ceph-mon[33846]: osdmap e142: 8 total, 8 up, 8 in 2024-03-21T12:55:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key1"}]: dispatch 2024-03-21T12:55:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/152823803' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-81636-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:43 smithi155 ceph-mon[33846]: from='client.24709 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key1"}]: dispatch 2024-03-21T12:55:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:43 smithi155 ceph-mon[33846]: from='client.25852 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-81636-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:43 smithi043 ceph-mon[30062]: from='client.24709 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-21T12:55:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:43 smithi043 ceph-mon[30062]: from='client.35861 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82022-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:43 smithi043 ceph-mon[30062]: osdmap e142: 8 total, 8 up, 8 in 2024-03-21T12:55:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key1"}]: dispatch 2024-03-21T12:55:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/152823803' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-81636-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:43 smithi043 ceph-mon[30062]: from='client.24709 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key1"}]: dispatch 2024-03-21T12:55:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:43 smithi043 ceph-mon[30062]: from='client.25852 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-81636-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:43 smithi043 ceph-mon[41699]: from='client.24709 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-21T12:55:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:43 smithi043 ceph-mon[41699]: from='client.35861 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-82022-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:43 smithi043 ceph-mon[41699]: osdmap e142: 8 total, 8 up, 8 in 2024-03-21T12:55:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1105226837' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key1"}]: dispatch 2024-03-21T12:55:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/152823803' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-81636-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:43 smithi043 ceph-mon[41699]: from='client.24709 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key1"}]: dispatch 2024-03-21T12:55:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:43 smithi043 ceph-mon[41699]: from='client.25852 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-81636-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:44.370 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-03-21T12:55:44.370 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-03-21T12:55:44.370 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-03-21T12:55:44.370 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-03-21T12:55:44.370 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-03-21T12:55:44.370 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-03-21T12:55:44.370 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-03-21T12:55:44.370 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-03-21T12:55:44.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: seed 81791 2024-03-21T12:55:44.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-03-21T12:55:44.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (1 ms) 2024-03-21T12:55:44.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-03-21T12:55:44.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (405 ms) 2024-03-21T12:55:44.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-03-21T12:55:44.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (12 ms) 2024-03-21T12:55:44.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-03-21T12:55:44.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (8 ms) 2024-03-21T12:55:44.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-03-21T12:55:44.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (9 ms) 2024-03-21T12:55:44.371 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-03-21T12:55:44.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (2 ms) 2024-03-21T12:55:44.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-03-21T12:55:44.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (2 ms) 2024-03-21T12:55:44.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-03-21T12:55:44.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (5 ms) 2024-03-21T12:55:44.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-03-21T12:55:44.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (3 ms) 2024-03-21T12:55:44.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-03-21T12:55:44.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (9 ms) 2024-03-21T12:55:44.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-03-21T12:55:44.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (3 ms) 2024-03-21T12:55:44.372 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-03-21T12:55:44.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (6 ms) 2024-03-21T12:55:44.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-03-21T12:55:44.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (8 ms) 2024-03-21T12:55:44.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-03-21T12:55:44.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: osd_max_attr_size = 0 2024-03-21T12:55:44.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-03-21T12:55:44.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (1616 ms) 2024-03-21T12:55:44.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-03-21T12:55:44.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (384 ms) 2024-03-21T12:55:44.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-03-21T12:55:44.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-03-21T12:55:44.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: done waiting, doing copies 2024-03-21T12:55:44.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: waiting for final deep scrubs... 2024-03-21T12:55:44.373 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: done waiting 2024-03-21T12:55:44.374 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60583 ms) 2024-03-21T12:55:44.374 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-03-21T12:55:44.374 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (4 ms) 2024-03-21T12:55:44.374 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-03-21T12:55:44.374 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (6 ms) 2024-03-21T12:55:44.374 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-03-21T12:55:44.374 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4415 ms) 2024-03-21T12:55:44.374 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-03-21T12:55:44.374 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-03-21T12:55:44.374 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-03-21T12:55:44.374 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-03-21T12:55:44.374 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-03-21T12:55:44.375 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-03-21T12:55:44.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:44 smithi155 ceph-mon[33846]: pgmap v176: 484 pgs: 32 creating+peering, 452 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-21T12:55:44.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:44 smithi155 ceph-mon[33846]: from='client.24709 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key1"}]': finished 2024-03-21T12:55:44.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:44 smithi155 ceph-mon[33846]: from='client.25852 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-81636-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:44.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:44 smithi155 ceph-mon[33846]: osdmap e143: 8 total, 8 up, 8 in 2024-03-21T12:55:44.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:44 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1151053303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-81661-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:44.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:44 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:44.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:44 smithi043 ceph-mon[30062]: pgmap v176: 484 pgs: 32 creating+peering, 452 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-21T12:55:44.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:44 smithi043 ceph-mon[30062]: from='client.24709 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key1"}]': finished 2024-03-21T12:55:44.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:44 smithi043 ceph-mon[30062]: from='client.25852 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-81636-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:44.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:44 smithi043 ceph-mon[30062]: osdmap e143: 8 total, 8 up, 8 in 2024-03-21T12:55:44.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:44 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1151053303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-81661-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:44.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:44 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:44.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:44 smithi043 ceph-mon[41699]: pgmap v176: 484 pgs: 32 creating+peering, 452 active+clean; 217 MiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-21T12:55:44.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:44 smithi043 ceph-mon[41699]: from='client.24709 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi043-81791-1","app":"app1","key":"key1"}]': finished 2024-03-21T12:55:44.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:44 smithi043 ceph-mon[41699]: from='client.25852 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi043-81636-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:44.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:44 smithi043 ceph-mon[41699]: osdmap e143: 8 total, 8 up, 8 in 2024-03-21T12:55:44.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:44 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1151053303' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-81661-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:44.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:44 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:45.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:45 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1151053303' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-81661-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:45.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:45 smithi155 ceph-mon[33846]: osdmap e144: 8 total, 8 up, 8 in 2024-03-21T12:55:45.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:45 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4266245853' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:45.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:45 smithi155 ceph-mon[33846]: from='client.25885 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:45.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:45 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4266245853' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:45.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:45 smithi155 ceph-mon[33846]: from='client.25885 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:45.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:45 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4266245853' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:55:45.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:45 smithi155 ceph-mon[33846]: from='client.25885 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:55:45.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:45 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1151053303' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-81661-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:45 smithi043 ceph-mon[30062]: osdmap e144: 8 total, 8 up, 8 in 2024-03-21T12:55:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4266245853' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:45 smithi043 ceph-mon[30062]: from='client.25885 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4266245853' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:45 smithi043 ceph-mon[30062]: from='client.25885 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4266245853' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:55:45.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:45 smithi043 ceph-mon[30062]: from='client.25885 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:55:45.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:45.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:45 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1151053303' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi043-81661-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:45.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:45 smithi043 ceph-mon[41699]: osdmap e144: 8 total, 8 up, 8 in 2024-03-21T12:55:45.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:45 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4266245853' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:45.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:45 smithi043 ceph-mon[41699]: from='client.25885 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:45.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:45 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4266245853' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:45.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:45 smithi043 ceph-mon[41699]: from='client.25885 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:45.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:45 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4266245853' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:55:45.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:45 smithi043 ceph-mon[41699]: from='client.25885 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:55:45.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:45 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:46.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:46 smithi155 ceph-mon[33846]: pgmap v179: 452 pgs: 32 unknown, 420 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-21T12:55:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:46 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:46 smithi155 ceph-mon[33846]: from='client.25885 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:55:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:46 smithi155 ceph-mon[33846]: osdmap e145: 8 total, 8 up, 8 in 2024-03-21T12:55:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2872227570' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-81636-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:46 smithi155 ceph-mon[33846]: from='client.25882 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-81636-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4266245853' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:46 smithi155 ceph-mon[33846]: from='client.25885 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1663443999' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:55:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:46 smithi155 ceph-mon[33846]: from='client.35912 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:55:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1663443999' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:55:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:46 smithi155 ceph-mon[33846]: from='client.35912 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:55:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1663443999' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82022-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:55:46.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:46 smithi155 ceph-mon[33846]: from='client.35912 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82022-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:55:46.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:46.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[41699]: pgmap v179: 452 pgs: 32 unknown, 420 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-21T12:55:46.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:46.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[41699]: from='client.25885 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:55:46.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[41699]: osdmap e145: 8 total, 8 up, 8 in 2024-03-21T12:55:46.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2872227570' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-81636-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:46.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[41699]: from='client.25882 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-81636-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:46.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4266245853' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:46.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[41699]: from='client.25885 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:46.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1663443999' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:55:46.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[41699]: from='client.35912 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:55:46.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1663443999' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:55:46.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[41699]: from='client.35912 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:55:46.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1663443999' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82022-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:55:46.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[41699]: from='client.35912 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82022-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:55:46.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:46.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[30062]: pgmap v179: 452 pgs: 32 unknown, 420 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-21T12:55:46.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:46.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[30062]: from='client.25885 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81791-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:55:46.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[30062]: osdmap e145: 8 total, 8 up, 8 in 2024-03-21T12:55:46.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2872227570' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-81636-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:46.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[30062]: from='client.25882 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-81636-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:46.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4266245853' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:46.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[30062]: from='client.25885 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:46.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1663443999' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:55:46.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[30062]: from='client.35912 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:55:46.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1663443999' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:55:46.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[30062]: from='client.35912 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:55:46.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1663443999' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82022-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:55:46.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[30062]: from='client.35912 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82022-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:55:46.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:47 smithi155 ceph-mon[33846]: from='client.25882 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-81636-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:47 smithi155 ceph-mon[33846]: from='client.35912 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82022-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:55:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1663443999' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-82022-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:55:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:47 smithi155 ceph-mon[33846]: osdmap e146: 8 total, 8 up, 8 in 2024-03-21T12:55:48.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:47 smithi155 ceph-mon[33846]: from='client.35912 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-82022-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:55:48.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2778713142' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-81661-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:48.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:48.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:47 smithi155 ceph-mon[33846]: from='client.25885 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81791-24"}]': finished 2024-03-21T12:55:48.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:55:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:55:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:55:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:47 smithi043 ceph-mon[30062]: from='client.25882 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-81636-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:47 smithi043 ceph-mon[30062]: from='client.35912 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82022-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:55:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1663443999' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-82022-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:55:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:47 smithi043 ceph-mon[30062]: osdmap e146: 8 total, 8 up, 8 in 2024-03-21T12:55:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:47 smithi043 ceph-mon[30062]: from='client.35912 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-82022-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:55:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2778713142' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-81661-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:47 smithi043 ceph-mon[30062]: from='client.25885 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81791-24"}]': finished 2024-03-21T12:55:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:47 smithi043 ceph-mon[41699]: from='client.25882 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi043-81636-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:47 smithi043 ceph-mon[41699]: from='client.35912 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82022-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:55:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1663443999' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-82022-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:55:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:47 smithi043 ceph-mon[41699]: osdmap e146: 8 total, 8 up, 8 in 2024-03-21T12:55:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:47 smithi043 ceph-mon[41699]: from='client.35912 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-82022-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:55:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2778713142' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-81661-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:47 smithi043 ceph-mon[41699]: from='client.25885 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81791-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81791-24"}]': finished 2024-03-21T12:55:48.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:55:48 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:55:49.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:48 smithi155 ceph-mon[33846]: pgmap v182: 452 pgs: 64 unknown, 388 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-21T12:55:49.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:48 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2778713142' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-81661-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:49.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:48 smithi155 ceph-mon[33846]: osdmap e147: 8 total, 8 up, 8 in 2024-03-21T12:55:49.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:48 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:55:49.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:48 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:49.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:48 smithi155 ceph-mon[33846]: from='client.35912 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-82022-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-82022-16"}]': finished 2024-03-21T12:55:49.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:48 smithi155 ceph-mon[33846]: osdmap e148: 8 total, 8 up, 8 in 2024-03-21T12:55:49.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:48 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4266245853' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81791-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:49.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:48 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/803096360' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-81636-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:49.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:48 smithi155 ceph-mon[33846]: from='client.25885 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81791-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:49.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:48 smithi155 ceph-mon[33846]: from='client.25912 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-81636-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:49.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:48 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:55:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:48 smithi043 ceph-mon[30062]: pgmap v182: 452 pgs: 64 unknown, 388 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-21T12:55:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:48 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2778713142' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-81661-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:48 smithi043 ceph-mon[30062]: osdmap e147: 8 total, 8 up, 8 in 2024-03-21T12:55:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:48 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:55:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:48 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:48 smithi043 ceph-mon[30062]: from='client.35912 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-82022-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-82022-16"}]': finished 2024-03-21T12:55:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:48 smithi043 ceph-mon[30062]: osdmap e148: 8 total, 8 up, 8 in 2024-03-21T12:55:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:48 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4266245853' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81791-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:48 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/803096360' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-81636-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:48 smithi043 ceph-mon[30062]: from='client.25885 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81791-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:48 smithi043 ceph-mon[30062]: from='client.25912 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-81636-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:49.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:48 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:55:49.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:48 smithi043 ceph-mon[41699]: pgmap v182: 452 pgs: 64 unknown, 388 active+clean; 592 KiB data, 1.2 GiB used, 714 GiB / 715 GiB avail 2024-03-21T12:55:49.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:48 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2778713142' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi043-81661-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:49.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:48 smithi043 ceph-mon[41699]: osdmap e147: 8 total, 8 up, 8 in 2024-03-21T12:55:49.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:48 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:55:49.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:48 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:49.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:48 smithi043 ceph-mon[41699]: from='client.35912 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi043-82022-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi043-82022-16"}]': finished 2024-03-21T12:55:49.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:48 smithi043 ceph-mon[41699]: osdmap e148: 8 total, 8 up, 8 in 2024-03-21T12:55:49.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:48 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4266245853' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81791-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:49.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:48 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/803096360' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-81636-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:49.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:48 smithi043 ceph-mon[41699]: from='client.25885 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81791-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:49.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:48 smithi043 ceph-mon[41699]: from='client.25912 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-81636-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:49.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:48 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:55:50.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:49 smithi155 ceph-mon[33846]: pgmap v185: 468 pgs: 8 creating+peering, 72 unknown, 388 active+clean; 592 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:55:50.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:49 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:50.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:49 smithi155 ceph-mon[33846]: from='client.25885 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81791-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:50.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:49 smithi155 ceph-mon[33846]: from='client.25912 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-81636-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:50.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:49 smithi155 ceph-mon[33846]: osdmap e149: 8 total, 8 up, 8 in 2024-03-21T12:55:50.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:49 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1395980209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-81661-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:50.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:49 smithi155 ceph-mon[33846]: from='client.35924 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-81661-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:50.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:49 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4266245853' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81791-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:50.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:49 smithi155 ceph-mon[33846]: from='client.25885 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81791-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:49 smithi043 ceph-mon[30062]: pgmap v185: 468 pgs: 8 creating+peering, 72 unknown, 388 active+clean; 592 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:55:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:49 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:49 smithi043 ceph-mon[30062]: from='client.25885 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81791-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:49 smithi043 ceph-mon[30062]: from='client.25912 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-81636-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:49 smithi043 ceph-mon[30062]: osdmap e149: 8 total, 8 up, 8 in 2024-03-21T12:55:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:49 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1395980209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-81661-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:49 smithi043 ceph-mon[30062]: from='client.35924 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-81661-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:49 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4266245853' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81791-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:49 smithi043 ceph-mon[30062]: from='client.25885 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81791-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:50.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:49 smithi043 ceph-mon[41699]: pgmap v185: 468 pgs: 8 creating+peering, 72 unknown, 388 active+clean; 592 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:55:50.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:49 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:50.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:49 smithi043 ceph-mon[41699]: from='client.25885 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81791-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:50.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:49 smithi043 ceph-mon[41699]: from='client.25912 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi043-81636-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:50.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:49 smithi043 ceph-mon[41699]: osdmap e149: 8 total, 8 up, 8 in 2024-03-21T12:55:50.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:49 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1395980209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-81661-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:50.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:49 smithi043 ceph-mon[41699]: from='client.35924 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-81661-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:50.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:49 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4266245853' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81791-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:50.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:49 smithi043 ceph-mon[41699]: from='client.25885 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81791-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:50 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:50 smithi155 ceph-mon[33846]: from='client.35924 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-81661-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:50 smithi155 ceph-mon[33846]: from='client.25885 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81791-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:50 smithi155 ceph-mon[33846]: osdmap e150: 8 total, 8 up, 8 in 2024-03-21T12:55:51.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:50 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:51.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:50 smithi043 ceph-mon[30062]: from='client.35924 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-81661-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:50 smithi043 ceph-mon[30062]: from='client.25885 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81791-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:50 smithi043 ceph-mon[30062]: osdmap e150: 8 total, 8 up, 8 in 2024-03-21T12:55:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:50 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:50 smithi043 ceph-mon[41699]: from='client.35924 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi043-81661-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:50 smithi043 ceph-mon[41699]: from='client.25885 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81791-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:50 smithi043 ceph-mon[41699]: osdmap e150: 8 total, 8 up, 8 in 2024-03-21T12:55:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:51 smithi155 ceph-mon[33846]: pgmap v188: 468 pgs: 8 creating+peering, 72 unknown, 388 active+clean; 592 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:55:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:51 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:51 smithi155 ceph-mon[33846]: osdmap e151: 8 total, 8 up, 8 in 2024-03-21T12:55:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:51 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4019603126' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-81636-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:51 smithi043 ceph-mon[30062]: pgmap v188: 468 pgs: 8 creating+peering, 72 unknown, 388 active+clean; 592 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:55:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:51 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:51 smithi043 ceph-mon[30062]: osdmap e151: 8 total, 8 up, 8 in 2024-03-21T12:55:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:51 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4019603126' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-81636-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:52.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:51 smithi043 ceph-mon[41699]: pgmap v188: 468 pgs: 8 creating+peering, 72 unknown, 388 active+clean; 592 KiB data, 581 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:55:52.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:51 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:52.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:51 smithi043 ceph-mon[41699]: osdmap e151: 8 total, 8 up, 8 in 2024-03-21T12:55:52.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:51 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4019603126' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-81636-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:53.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:52 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:53.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:53.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4019603126' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-81636-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:53.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:52 smithi155 ceph-mon[33846]: osdmap e152: 8 total, 8 up, 8 in 2024-03-21T12:55:53.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1347178560' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-81661-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:53.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:52 smithi155 ceph-mon[33846]: from='client.35936 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-81661-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:53.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4266245853' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:53.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:52 smithi155 ceph-mon[33846]: from='client.25885 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:52 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4019603126' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-81636-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:52 smithi043 ceph-mon[30062]: osdmap e152: 8 total, 8 up, 8 in 2024-03-21T12:55:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1347178560' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-81661-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:52 smithi043 ceph-mon[30062]: from='client.35936 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-81661-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4266245853' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:53.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:52 smithi043 ceph-mon[30062]: from='client.25885 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:53.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:52 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:53.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:53.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4019603126' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi043-81636-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:53.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:52 smithi043 ceph-mon[41699]: osdmap e152: 8 total, 8 up, 8 in 2024-03-21T12:55:53.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1347178560' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-81661-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:53.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:52 smithi043 ceph-mon[41699]: from='client.35936 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-81661-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:53.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4266245853' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:53.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:52 smithi043 ceph-mon[41699]: from='client.25885 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:53.388 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP (6 api_snapshots_pp: Running main() from gmock_main.cc 2024-03-21T12:55:53.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [==========] Running 21 tests from 5 test suites. 2024-03-21T12:55:53.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-03-21T12:55:53.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-03-21T12:55:53.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: seed 82022 2024-03-21T12:55:53.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-03-21T12:55:53.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (4049 ms) 2024-03-21T12:55:53.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-03-21T12:55:53.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (4284 ms) 2024-03-21T12:55:53.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-03-21T12:55:53.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (5098 ms) 2024-03-21T12:55:53.389 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-03-21T12:55:53.390 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (2136 ms) 2024-03-21T12:55:53.390 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-03-21T12:55:53.390 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (6852 ms) 2024-03-21T12:55:53.390 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (22419 ms total) 2024-03-21T12:55:53.390 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: 2024-03-21T12:55:53.390 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-03-21T12:55:53.390 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-03-21T12:55:53.390 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (4245 ms) 2024-03-21T12:55:53.390 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-03-21T12:55:53.390 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (3659 ms) 2024-03-21T12:55:53.390 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-03-21T12:55:53.390 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (6921 ms) 2024-03-21T12:55:53.390 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-03-21T12:55:53.391 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (3998 ms) 2024-03-21T12:55:53.391 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-03-21T12:55:53.391 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2055 ms) 2024-03-21T12:55:53.391 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-03-21T12:55:53.391 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-2267-g960fbbe4/rpm/el9/BUILD/ceph-19.0.0-2267-g960fbbe4/src/test/librados/snapshots_cxx.cc:463: Skipped 2024-03-21T12:55:53.391 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: 2024-03-21T12:55:53.391 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2024-03-21T12:55:53.391 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-03-21T12:55:53.391 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: deleting snap 14 in pool LibRadosSnapshotsSelfManagedPP_smithi043-82022-7 2024-03-21T12:55:53.391 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: waiting for snaps to purge 2024-03-21T12:55:53.391 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17950 ms) 2024-03-21T12:55:53.391 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP (38828 ms total) 2024-03-21T12:55:53.391 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: 2024-03-21T12:55:53.392 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-03-21T12:55:53.392 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-03-21T12:55:53.392 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (8 ms) 2024-03-21T12:55:53.392 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-03-21T12:55:53.392 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5060 ms) 2024-03-21T12:55:53.392 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5068 ms total) 2024-03-21T12:55:53.392 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: 2024-03-21T12:55:53.392 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-03-21T12:55:53.392 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-03-21T12:55:53.392 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (3006 ms) 2024-03-21T12:55:53.392 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-03-21T12:55:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:53 smithi155 ceph-mon[33846]: pgmap v191: 460 pgs: 32 creating+peering, 428 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-21T12:55:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:53 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:55:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:53 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:53 smithi155 ceph-mon[33846]: from='client.35936 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-81661-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:53 smithi155 ceph-mon[33846]: from='client.25885 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81791-24"}]': finished 2024-03-21T12:55:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:53 smithi155 ceph-mon[33846]: osdmap e153: 8 total, 8 up, 8 in 2024-03-21T12:55:54.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:53 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4266245853' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:54.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:53 smithi155 ceph-mon[33846]: from='client.25885 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:53 smithi043 ceph-mon[30062]: pgmap v191: 460 pgs: 32 creating+peering, 428 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-21T12:55:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:53 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:55:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:53 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:53 smithi043 ceph-mon[30062]: from='client.35936 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-81661-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:53 smithi043 ceph-mon[30062]: from='client.25885 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81791-24"}]': finished 2024-03-21T12:55:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:53 smithi043 ceph-mon[30062]: osdmap e153: 8 total, 8 up, 8 in 2024-03-21T12:55:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:53 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4266245853' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:53 smithi043 ceph-mon[30062]: from='client.25885 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:54.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:53 smithi043 ceph-mon[41699]: pgmap v191: 460 pgs: 32 creating+peering, 428 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-21T12:55:54.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:53 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:55:54.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:53 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:54.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:53 smithi043 ceph-mon[41699]: from='client.35936 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi043-81661-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:54.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:53 smithi043 ceph-mon[41699]: from='client.25885 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81791-24"}]': finished 2024-03-21T12:55:54.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:53 smithi043 ceph-mon[41699]: osdmap e153: 8 total, 8 up, 8 in 2024-03-21T12:55:54.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:53 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4266245853' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:54.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:53 smithi043 ceph-mon[41699]: from='client.25885 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81791-24"}]: dispatch 2024-03-21T12:55:55.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:54 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:55:55.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:54 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:55:55.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:54 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:55:55.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:54 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:55.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:54 smithi155 ceph-mon[33846]: from='client.25885 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81791-24"}]': finished 2024-03-21T12:55:55.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:54 smithi155 ceph-mon[33846]: osdmap e154: 8 total, 8 up, 8 in 2024-03-21T12:55:55.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:54 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/373513707' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-81636-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:54 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:55:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:54 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:55:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:54 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:55:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:54 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:54 smithi043 ceph-mon[30062]: from='client.25885 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81791-24"}]': finished 2024-03-21T12:55:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:54 smithi043 ceph-mon[30062]: osdmap e154: 8 total, 8 up, 8 in 2024-03-21T12:55:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:54 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/373513707' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-81636-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:55.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:54 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:55:55.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:54 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:55:55.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:54 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:55:55.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:54 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:55.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:54 smithi043 ceph-mon[41699]: from='client.25885 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81791-24"}]': finished 2024-03-21T12:55:55.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:54 smithi043 ceph-mon[41699]: osdmap e154: 8 total, 8 up, 8 in 2024-03-21T12:55:55.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:54 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/373513707' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-81636-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:55 smithi155 ceph-mon[33846]: pgmap v193: 428 pgs: 32 creating+peering, 396 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 574 B/s wr, 1 op/s 2024-03-21T12:55:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:55 smithi155 ceph-mon[33846]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 2 stray daemon(s) not managed by cephadm) 2024-03-21T12:55:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:55 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:55 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/373513707' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-81636-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:56.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:55 smithi155 ceph-mon[33846]: osdmap e155: 8 total, 8 up, 8 in 2024-03-21T12:55:56.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:55 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2932637533' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81661-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:56.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:55 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/45010207' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-81791-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:56.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:55 smithi155 ceph-mon[33846]: from='client.25939 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81661-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:56.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:55 smithi155 ceph-mon[33846]: from='client.25942 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-81791-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:56.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:55 smithi043 ceph-mon[30062]: pgmap v193: 428 pgs: 32 creating+peering, 396 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 574 B/s wr, 1 op/s 2024-03-21T12:55:56.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:55 smithi043 ceph-mon[30062]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 2 stray daemon(s) not managed by cephadm) 2024-03-21T12:55:56.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:55 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:56.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:55 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/373513707' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-81636-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:56.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:55 smithi043 ceph-mon[30062]: osdmap e155: 8 total, 8 up, 8 in 2024-03-21T12:55:56.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:55 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2932637533' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81661-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:56.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:55 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/45010207' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-81791-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:56.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:55 smithi043 ceph-mon[30062]: from='client.25939 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81661-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:56.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:55 smithi043 ceph-mon[30062]: from='client.25942 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-81791-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:56.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:55 smithi043 ceph-mon[41699]: pgmap v193: 428 pgs: 32 creating+peering, 396 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 574 B/s wr, 1 op/s 2024-03-21T12:55:56.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:55 smithi043 ceph-mon[41699]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 2 stray daemon(s) not managed by cephadm) 2024-03-21T12:55:56.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:55 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:56.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:55 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/373513707' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi043-81636-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:56.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:55 smithi043 ceph-mon[41699]: osdmap e155: 8 total, 8 up, 8 in 2024-03-21T12:55:56.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:55 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2932637533' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81661-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:56.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:55 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/45010207' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-81791-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:56.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:55 smithi043 ceph-mon[41699]: from='client.25939 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81661-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:56.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:55 smithi043 ceph-mon[41699]: from='client.25942 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-81791-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:56 smithi155 ceph-mon[33846]: pgmap v196: 492 pgs: 96 unknown, 396 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 575 B/s wr, 1 op/s 2024-03-21T12:55:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:56 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:56 smithi155 ceph-mon[33846]: from='client.25939 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81661-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:56 smithi155 ceph-mon[33846]: from='client.25942 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-81791-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:56 smithi155 ceph-mon[33846]: osdmap e156: 8 total, 8 up, 8 in 2024-03-21T12:55:57.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:56 smithi043 ceph-mon[30062]: pgmap v196: 492 pgs: 96 unknown, 396 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 575 B/s wr, 1 op/s 2024-03-21T12:55:57.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:56 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:57.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:56 smithi043 ceph-mon[30062]: from='client.25939 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81661-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:57.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:56 smithi043 ceph-mon[30062]: from='client.25942 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-81791-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:57.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:56 smithi043 ceph-mon[30062]: osdmap e156: 8 total, 8 up, 8 in 2024-03-21T12:55:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:56 smithi043 ceph-mon[41699]: pgmap v196: 492 pgs: 96 unknown, 396 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 575 B/s wr, 1 op/s 2024-03-21T12:55:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:56 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:56 smithi043 ceph-mon[41699]: from='client.25939 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi043-81661-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:56 smithi043 ceph-mon[41699]: from='client.25942 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi043-81791-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:57.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:56 smithi043 ceph-mon[41699]: osdmap e156: 8 total, 8 up, 8 in 2024-03-21T12:55:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:57 smithi155 ceph-mon[33846]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:57 smithi155 ceph-mon[33846]: osdmap e157: 8 total, 8 up, 8 in 2024-03-21T12:55:58.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:57 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3102867367' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-81636-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:58.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:57 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:58.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:55:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:55:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:55:58.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:57 smithi043 ceph-mon[30062]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:58.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:57 smithi043 ceph-mon[30062]: osdmap e157: 8 total, 8 up, 8 in 2024-03-21T12:55:58.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:57 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3102867367' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-81636-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:58.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:57 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:58.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:57 smithi043 ceph-mon[41699]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:55:58.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:57 smithi043 ceph-mon[41699]: osdmap e157: 8 total, 8 up, 8 in 2024-03-21T12:55:58.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:57 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3102867367' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-81636-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:58.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:57 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:58.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:55:58 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:55:59.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:59 smithi155 ceph-mon[33846]: pgmap v199: 428 pgs: 32 unknown, 396 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:55:59.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:59 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:55:59.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:59 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3102867367' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-81636-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:59.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:59 smithi155 ceph-mon[33846]: osdmap e158: 8 total, 8 up, 8 in 2024-03-21T12:55:59.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:59 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1663443999' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:55:59.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:59 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1048366964' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-81661-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:59.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:59 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3763196037' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-81791-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:59.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:59 smithi155 ceph-mon[33846]: from='client.35912 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:55:59.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:59 smithi155 ceph-mon[33846]: from='client.25957 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-81661-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:59.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:59 smithi155 ceph-mon[33846]: from='client.25960 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-81791-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:59.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:55:59 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:59 smithi043 ceph-mon[41699]: pgmap v199: 428 pgs: 32 unknown, 396 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:55:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:59 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:55:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:59 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3102867367' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-81636-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:59 smithi043 ceph-mon[41699]: osdmap e158: 8 total, 8 up, 8 in 2024-03-21T12:55:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:59 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1663443999' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:55:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:59 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1048366964' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-81661-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:59 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3763196037' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-81791-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:59 smithi043 ceph-mon[41699]: from='client.35912 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:55:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:59 smithi043 ceph-mon[41699]: from='client.25957 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-81661-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:59 smithi043 ceph-mon[41699]: from='client.25960 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-81791-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:55:59 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:55:59.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:59 smithi043 ceph-mon[30062]: pgmap v199: 428 pgs: 32 unknown, 396 active+clean; 592 KiB data, 604 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:55:59.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:59 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:55:59.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:59 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3102867367' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi043-81636-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:55:59.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:59 smithi043 ceph-mon[30062]: osdmap e158: 8 total, 8 up, 8 in 2024-03-21T12:55:59.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:59 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1663443999' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:55:59.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:59 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1048366964' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-81661-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:59.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:59 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3763196037' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-81791-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:59.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:59 smithi043 ceph-mon[30062]: from='client.35912 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:55:59.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:59 smithi043 ceph-mon[30062]: from='client.25957 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-81661-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:59.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:59 smithi043 ceph-mon[30062]: from='client.25960 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-81791-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:55:59.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:55:59 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:00.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:00 smithi155 ceph-mon[33846]: from='client.35912 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82022-16"}]': finished 2024-03-21T12:56:00.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:00 smithi155 ceph-mon[33846]: from='client.25957 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-81661-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:00.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:00 smithi155 ceph-mon[33846]: from='client.25960 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-81791-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:00 smithi155 ceph-mon[33846]: osdmap e159: 8 total, 8 up, 8 in 2024-03-21T12:56:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:00 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1663443999' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:56:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:00 smithi155 ceph-mon[33846]: from='client.35912 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:56:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:00 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:00 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1455814188' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:00 smithi155 ceph-mon[33846]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:00 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1455814188' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:00 smithi155 ceph-mon[33846]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:00 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1455814188' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-81636-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:00 smithi155 ceph-mon[33846]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-81636-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:00 smithi155 ceph-mon[33846]: from='client.35912 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82022-16"}]': finished 2024-03-21T12:56:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:00 smithi155 ceph-mon[33846]: from='client.35972 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-81636-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:00 smithi155 ceph-mon[33846]: osdmap e160: 8 total, 8 up, 8 in 2024-03-21T12:56:00.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[30062]: from='client.35912 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82022-16"}]': finished 2024-03-21T12:56:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[30062]: from='client.25957 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-81661-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[30062]: from='client.25960 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-81791-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[30062]: osdmap e159: 8 total, 8 up, 8 in 2024-03-21T12:56:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1663443999' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:56:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[30062]: from='client.35912 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:56:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1455814188' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[30062]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1455814188' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[30062]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1455814188' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-81636-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[30062]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-81636-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[30062]: from='client.35912 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82022-16"}]': finished 2024-03-21T12:56:00.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[30062]: from='client.35972 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-81636-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:00.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[30062]: osdmap e160: 8 total, 8 up, 8 in 2024-03-21T12:56:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[41699]: from='client.35912 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi043-82022-16"}]': finished 2024-03-21T12:56:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[41699]: from='client.25957 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi043-81661-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[41699]: from='client.25960 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi043-81791-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[41699]: osdmap e159: 8 total, 8 up, 8 in 2024-03-21T12:56:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1663443999' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:56:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[41699]: from='client.35912 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82022-16"}]: dispatch 2024-03-21T12:56:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1455814188' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[41699]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1455814188' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[41699]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:00.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1455814188' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-81636-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:00.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[41699]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-81636-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:00.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[41699]: from='client.35912 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi043-82022-16"}]': finished 2024-03-21T12:56:00.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[41699]: from='client.35972 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi043-81636-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:00.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:00 smithi043 ceph-mon[41699]: osdmap e160: 8 total, 8 up, 8 in 2024-03-21T12:56:01.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:01 smithi155 ceph-mon[33846]: pgmap v202: 452 pgs: 64 unknown, 388 active+clean; 592 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:56:01.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:01 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1455814188' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-81636-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:01.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:01 smithi155 ceph-mon[33846]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-81636-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:01.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:01 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:01.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:01 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1860810006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:01.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:01 smithi155 ceph-mon[33846]: from='client.25981 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:01.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:01 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1860810006' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:01.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:01 smithi155 ceph-mon[33846]: from='client.25981 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:01.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:01 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1860810006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:01.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:01 smithi155 ceph-mon[33846]: from='client.25981 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:01.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:01 smithi155 ceph-mon[33846]: from='client.25981 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:01.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:01 smithi043 ceph-mon[30062]: pgmap v202: 452 pgs: 64 unknown, 388 active+clean; 592 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:56:01.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:01 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1455814188' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-81636-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:01.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:01 smithi043 ceph-mon[30062]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-81636-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:01.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:01 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:01.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:01 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1860810006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:01.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:01 smithi043 ceph-mon[30062]: from='client.25981 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:01.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:01 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1860810006' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:01.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:01 smithi043 ceph-mon[30062]: from='client.25981 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:01.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:01 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1860810006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:01.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:01 smithi043 ceph-mon[30062]: from='client.25981 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:01.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:01 smithi043 ceph-mon[30062]: from='client.25981 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:01.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:01 smithi043 ceph-mon[41699]: pgmap v202: 452 pgs: 64 unknown, 388 active+clean; 592 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:56:01.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:01 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1455814188' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-81636-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:01.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:01 smithi043 ceph-mon[41699]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-81636-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:01.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:01 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:01.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:01 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1860810006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:01.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:01 smithi043 ceph-mon[41699]: from='client.25981 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:01.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:01 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1860810006' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:01.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:01 smithi043 ceph-mon[41699]: from='client.25981 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:01.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:01 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1860810006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:01.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:01 smithi043 ceph-mon[41699]: from='client.25981 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:01.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:01 smithi043 ceph-mon[41699]: from='client.25981 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:02.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:02 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1860810006' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:02.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:02 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:02.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:02 smithi155 ceph-mon[33846]: osdmap e161: 8 total, 8 up, 8 in 2024-03-21T12:56:02.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:02 smithi155 ceph-mon[33846]: from='client.25981 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:02.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:02 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3230143627' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-81661-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:02.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:02 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3901324556' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-81791-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:02.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:02 smithi155 ceph-mon[33846]: from='client.25978 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-81661-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:02.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:02 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:02.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:02 smithi155 ceph-mon[33846]: from='client.35972 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-81636-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-81636-27"}]': finished 2024-03-21T12:56:02.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:02 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3901324556' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-81791-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:02.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:02 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1860810006' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:02.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:02 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:02.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:02 smithi043 ceph-mon[30062]: osdmap e161: 8 total, 8 up, 8 in 2024-03-21T12:56:02.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:02 smithi043 ceph-mon[30062]: from='client.25981 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:02.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:02 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3230143627' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-81661-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:02.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:02 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3901324556' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-81791-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:02.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:02 smithi043 ceph-mon[30062]: from='client.25978 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-81661-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:02.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:02 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:02.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:02 smithi043 ceph-mon[30062]: from='client.35972 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-81636-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-81636-27"}]': finished 2024-03-21T12:56:02.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:02 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3901324556' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-81791-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:02.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:02 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1860810006' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:02.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:02 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:02.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:02 smithi043 ceph-mon[41699]: osdmap e161: 8 total, 8 up, 8 in 2024-03-21T12:56:02.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:02 smithi043 ceph-mon[41699]: from='client.25981 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:02.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:02 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3230143627' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-81661-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:02.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:02 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3901324556' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-81791-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:02.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:02 smithi043 ceph-mon[41699]: from='client.25978 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-81661-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:02.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:02 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:02.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:02 smithi043 ceph-mon[41699]: from='client.35972 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi043-81636-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi043-81636-27"}]': finished 2024-03-21T12:56:02.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:02 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3901324556' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi043-81791-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:03.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:03 smithi155 ceph-mon[33846]: pgmap v205: 452 pgs: 64 unknown, 388 active+clean; 592 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:56:03.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:03 smithi155 ceph-mon[33846]: from='client.25978 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-81661-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:03.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:03 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:03.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:03 smithi155 ceph-mon[33846]: osdmap e162: 8 total, 8 up, 8 in 2024-03-21T12:56:03.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:03 smithi155 ceph-mon[33846]: from='client.25981 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]': finished 2024-03-21T12:56:03.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:03 smithi155 ceph-mon[33846]: osdmap e163: 8 total, 8 up, 8 in 2024-03-21T12:56:03.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:03 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:03.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:03 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4037723181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:03.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:03 smithi155 ceph-mon[33846]: from='client.25987 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:03.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:03 smithi043 ceph-mon[30062]: pgmap v205: 452 pgs: 64 unknown, 388 active+clean; 592 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:56:03.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:03 smithi043 ceph-mon[30062]: from='client.25978 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-81661-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:03.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:03 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:03.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:03 smithi043 ceph-mon[30062]: osdmap e162: 8 total, 8 up, 8 in 2024-03-21T12:56:03.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:03 smithi043 ceph-mon[30062]: from='client.25981 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]': finished 2024-03-21T12:56:03.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:03 smithi043 ceph-mon[30062]: osdmap e163: 8 total, 8 up, 8 in 2024-03-21T12:56:03.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:03 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:03.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:03 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4037723181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:03.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:03 smithi043 ceph-mon[30062]: from='client.25987 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:03.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:03 smithi043 ceph-mon[41699]: pgmap v205: 452 pgs: 64 unknown, 388 active+clean; 592 KiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:56:03.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:03 smithi043 ceph-mon[41699]: from='client.25978 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi043-81661-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:03.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:03 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:03.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:03 smithi043 ceph-mon[41699]: osdmap e162: 8 total, 8 up, 8 in 2024-03-21T12:56:03.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:03 smithi043 ceph-mon[41699]: from='client.25981 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]': finished 2024-03-21T12:56:03.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:03 smithi043 ceph-mon[41699]: osdmap e163: 8 total, 8 up, 8 in 2024-03-21T12:56:03.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:03 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:03.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:03 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4037723181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:03.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:03 smithi043 ceph-mon[41699]: from='client.25987 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:04.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4037723181' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:04.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:04.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:04 smithi155 ceph-mon[33846]: from='client.25987 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:04.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:04 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:04.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4037723181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-81791-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:04.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:04 smithi155 ceph-mon[33846]: from='client.25987 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-81791-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:04.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:04 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:56:04.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:04 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:56:04.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:04 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:56:04.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:04 smithi155 ceph-mon[33846]: from='client.25987 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-81791-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:04.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-13"}]: dispatch 2024-03-21T12:56:04.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:04 smithi155 ceph-mon[33846]: osdmap e164: 8 total, 8 up, 8 in 2024-03-21T12:56:04.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:04 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-13"}]: dispatch 2024-03-21T12:56:04.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2364797635' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-81661-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:04.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4037723181' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-81791-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:04.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:04 smithi155 ceph-mon[33846]: from='client.25987 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-81791-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:04.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1455814188' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:04.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:04 smithi155 ceph-mon[33846]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:04.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:04.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4037723181' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:04.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:04.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[30062]: from='client.25987 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:04.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:04.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4037723181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-81791-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:04.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[30062]: from='client.25987 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-81791-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:04.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:56:04.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:56:04.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:56:04.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[30062]: from='client.25987 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-81791-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:04.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-13"}]: dispatch 2024-03-21T12:56:04.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[30062]: osdmap e164: 8 total, 8 up, 8 in 2024-03-21T12:56:04.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-13"}]: dispatch 2024-03-21T12:56:04.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2364797635' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-81661-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:04.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4037723181' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-81791-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:04.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[30062]: from='client.25987 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-81791-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:04.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1455814188' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:04.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[30062]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:04.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:04.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4037723181' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:04.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:04.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[41699]: from='client.25987 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:04.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:04.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4037723181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-81791-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:04.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[41699]: from='client.25987 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-81791-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:04.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:56:04.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:56:04.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:56:04.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[41699]: from='client.25987 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi043-81791-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:04.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-13"}]: dispatch 2024-03-21T12:56:04.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[41699]: osdmap e164: 8 total, 8 up, 8 in 2024-03-21T12:56:04.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-13"}]: dispatch 2024-03-21T12:56:04.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2364797635' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-81661-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:04.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4037723181' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-81791-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:04.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[41699]: from='client.25987 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-81791-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:04.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1455814188' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:04.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[41699]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:04.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:05.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:05 smithi155 ceph-mon[33846]: pgmap v208: 404 pgs: 8 creating+peering, 8 unknown, 388 active+clean; 588 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:56:05.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:05 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:56:05.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:05 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-13"}]': finished 2024-03-21T12:56:05.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:05 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2364797635' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-81661-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:05.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:05 smithi155 ceph-mon[33846]: from='client.35972 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81636-27"}]': finished 2024-03-21T12:56:05.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:05 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1455814188' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:05.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:05 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:05.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:05 smithi155 ceph-mon[33846]: osdmap e165: 8 total, 8 up, 8 in 2024-03-21T12:56:05.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:05 smithi155 ceph-mon[33846]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:05 smithi043 ceph-mon[30062]: pgmap v208: 404 pgs: 8 creating+peering, 8 unknown, 388 active+clean; 588 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:56:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:05 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:56:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:05 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-13"}]': finished 2024-03-21T12:56:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:05 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2364797635' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-81661-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:05 smithi043 ceph-mon[30062]: from='client.35972 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81636-27"}]': finished 2024-03-21T12:56:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:05 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1455814188' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:05 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:05 smithi043 ceph-mon[30062]: osdmap e165: 8 total, 8 up, 8 in 2024-03-21T12:56:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:05 smithi043 ceph-mon[30062]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:05 smithi043 ceph-mon[41699]: pgmap v208: 404 pgs: 8 creating+peering, 8 unknown, 388 active+clean; 588 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:56:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:05 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:56:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:05 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-13"}]': finished 2024-03-21T12:56:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:05 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2364797635' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi043-81661-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:05.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:05 smithi043 ceph-mon[41699]: from='client.35972 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi043-81636-27"}]': finished 2024-03-21T12:56:05.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:05 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1455814188' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:05.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:05 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:05.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:05 smithi043 ceph-mon[41699]: osdmap e165: 8 total, 8 up, 8 in 2024-03-21T12:56:05.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:05 smithi043 ceph-mon[41699]: from='client.35972 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81636-27"}]: dispatch 2024-03-21T12:56:06.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosS api_aio: Running main() from gmock_main.cc 2024-03-21T12:56:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [==========] Running 42 tests from 2 test suites. 2024-03-21T12:56:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [----------] Global test environment set-up. 2024-03-21T12:56:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [----------] 26 tests from LibRadosAio 2024-03-21T12:56:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-03-21T12:56:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.TooBig (10925 ms) 2024-03-21T12:56:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-03-21T12:56:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (5625 ms) 2024-03-21T12:56:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-03-21T12:56:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (4988 ms) 2024-03-21T12:56:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-03-21T12:56:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (4316 ms) 2024-03-21T12:56:06.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-03-21T12:56:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (3953 ms) 2024-03-21T12:56:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-03-21T12:56:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (2905 ms) 2024-03-21T12:56:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-03-21T12:56:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (4179 ms) 2024-03-21T12:56:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-03-21T12:56:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (3174 ms) 2024-03-21T12:56:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-03-21T12:56:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (2637 ms) 2024-03-21T12:56:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-03-21T12:56:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (3152 ms) 2024-03-21T12:56:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-03-21T12:56:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (3119 ms) 2024-03-21T12:56:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-03-21T12:56:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (2734 ms) 2024-03-21T12:56:06.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-03-21T12:56:06.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (3007 ms) 2024-03-21T12:56:06.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-03-21T12:56:06.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (2995 ms) 2024-03-21T12:56:06.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-03-21T12:56:06.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.Flush (3014 ms) 2024-03-21T12:56:06.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-03-21T12:56:06.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (3261 ms) 2024-03-21T12:56:06.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-03-21T12:56:06.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (2734 ms) 2024-03-21T12:56:06.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-03-21T12:56:06.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (2999 ms) 2024-03-21T12:56:06.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-03-21T12:56:06.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3011 ms) 2024-03-21T12:56:06.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-03-21T12:56:06.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3034 ms) 2024-03-21T12:56:06.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.Operate2Mtime 2024-03-21T12:56:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.Operate2Mtime (3006 ms) 2024-03-21T12:56:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-03-21T12:56:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3269 ms) 2024-03-21T12:56:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-03-21T12:56:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (2745 ms) 2024-03-21T12:56:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-03-21T12:56:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3006 ms) 2024-03-21T12:56:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-03-21T12:56:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (3008 ms) 2024-03-21T12:56:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-03-21T12:56:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (2922 ms) 2024-03-21T12:56:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [----------] 26 tests from LibRadosAio (93719 ms total) 2024-03-21T12:56:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: 2024-03-21T12:56:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-03-21T12:56:06.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-03-21T12:56:07.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:07 smithi155 ceph-mon[33846]: pgmap v211: 428 pgs: 40 unknown, 388 active+clean; 588 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:56:07.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:07 smithi155 ceph-mon[33846]: from='client.25987 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-81791-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-81791-36"}]': finished 2024-03-21T12:56:07.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:07 smithi155 ceph-mon[33846]: from='client.35972 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81636-27"}]': finished 2024-03-21T12:56:07.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:07 smithi155 ceph-mon[33846]: osdmap e166: 8 total, 8 up, 8 in 2024-03-21T12:56:07.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:07 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:07.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:07 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4260480745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:07.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:07 smithi155 ceph-mon[33846]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:07.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:07 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4260480745' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:07.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:07 smithi155 ceph-mon[33846]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:07.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:07 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4260480745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-81636-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:07.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:07 smithi155 ceph-mon[33846]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-81636-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:07.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:07 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:07.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:07 smithi155 ceph-mon[33846]: from='client.25993 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-81636-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:07.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:07 smithi155 ceph-mon[33846]: osdmap e167: 8 total, 8 up, 8 in 2024-03-21T12:56:07.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:07 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:07.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:07 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4260480745' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-81636-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:07.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:07 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:07.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:07 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1719276545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-81661-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:07.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:07 smithi155 ceph-mon[33846]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-81636-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:07.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:07 smithi155 ceph-mon[33846]: from='client.25999 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-81661-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[30062]: pgmap v211: 428 pgs: 40 unknown, 388 active+clean; 588 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:56:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[30062]: from='client.25987 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-81791-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-81791-36"}]': finished 2024-03-21T12:56:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[30062]: from='client.35972 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81636-27"}]': finished 2024-03-21T12:56:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[30062]: osdmap e166: 8 total, 8 up, 8 in 2024-03-21T12:56:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:07.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4260480745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:07.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[30062]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:07.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4260480745' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:07.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[30062]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:07.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4260480745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-81636-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:07.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[30062]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-81636-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:07.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:07.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[30062]: from='client.25993 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-81636-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:07.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[30062]: osdmap e167: 8 total, 8 up, 8 in 2024-03-21T12:56:07.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:07.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4260480745' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-81636-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:07.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:07.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1719276545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-81661-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:07.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[30062]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-81636-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:07.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[30062]: from='client.25999 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-81661-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:07.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[41699]: pgmap v211: 428 pgs: 40 unknown, 388 active+clean; 588 KiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:56:07.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[41699]: from='client.25987 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi043-81791-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi043-81791-36"}]': finished 2024-03-21T12:56:07.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[41699]: from='client.35972 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi043-81636-27"}]': finished 2024-03-21T12:56:07.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[41699]: osdmap e166: 8 total, 8 up, 8 in 2024-03-21T12:56:07.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:07.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4260480745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:07.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[41699]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:07.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4260480745' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:07.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[41699]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:07.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4260480745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-81636-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:07.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[41699]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-81636-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:07.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:07.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[41699]: from='client.25993 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi043-81636-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:07.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[41699]: osdmap e167: 8 total, 8 up, 8 in 2024-03-21T12:56:07.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:07.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4260480745' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-81636-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:07.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:07.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1719276545' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-81661-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:07.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[41699]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-81636-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:07.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:07 smithi043 ceph-mon[41699]: from='client.25999 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-81661-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:08.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:56:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:56:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:56:08.594 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:56:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:56:08.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:08 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:08.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:08 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:08.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:08 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:08.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:08 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:08.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:08 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:08.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:08 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:09 smithi155 ceph-mon[33846]: pgmap v214: 436 pgs: 80 unknown, 356 active+clean; 586 KiB data, 610 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:56:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:09 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:56:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:09 smithi155 ceph-mon[33846]: from='client.25999 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-81661-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:09 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:09.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:09 smithi155 ceph-mon[33846]: osdmap e168: 8 total, 8 up, 8 in 2024-03-21T12:56:09.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:09 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4037723181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:09.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:09 smithi155 ceph-mon[33846]: from='client.25987 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:09.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:09 smithi155 ceph-mon[33846]: from='client.25993 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-81636-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-81636-28"}]': finished 2024-03-21T12:56:09.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:09 smithi155 ceph-mon[33846]: from='client.25987 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81791-36"}]': finished 2024-03-21T12:56:09.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:09 smithi155 ceph-mon[33846]: osdmap e169: 8 total, 8 up, 8 in 2024-03-21T12:56:09.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:09 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4037723181' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:09.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:09 smithi155 ceph-mon[33846]: from='client.25987 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:09.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:09 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:09.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:09 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[30062]: pgmap v214: 436 pgs: 80 unknown, 356 active+clean; 586 KiB data, 610 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:56:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:56:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[30062]: from='client.25999 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-81661-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[30062]: osdmap e168: 8 total, 8 up, 8 in 2024-03-21T12:56:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4037723181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[30062]: from='client.25987 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[30062]: from='client.25993 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-81636-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-81636-28"}]': finished 2024-03-21T12:56:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[30062]: from='client.25987 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81791-36"}]': finished 2024-03-21T12:56:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[30062]: osdmap e169: 8 total, 8 up, 8 in 2024-03-21T12:56:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4037723181' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:09.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[30062]: from='client.25987 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:09.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:09.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:09.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[41699]: pgmap v214: 436 pgs: 80 unknown, 356 active+clean; 586 KiB data, 610 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:56:09.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:56:09.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[41699]: from='client.25999 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi043-81661-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:09.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:09.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[41699]: osdmap e168: 8 total, 8 up, 8 in 2024-03-21T12:56:09.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4037723181' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:09.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[41699]: from='client.25987 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:09.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[41699]: from='client.25993 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi043-81636-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi043-81636-28"}]': finished 2024-03-21T12:56:09.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[41699]: from='client.25987 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi043-81791-36"}]': finished 2024-03-21T12:56:09.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[41699]: osdmap e169: 8 total, 8 up, 8 in 2024-03-21T12:56:09.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4037723181' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:09.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[41699]: from='client.25987 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81791-36"}]: dispatch 2024-03-21T12:56:09.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:09.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:09 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:10.335 INFO:tasks.workunit.client.0.smithi043.stdout:7481 ms total) 2024-03-21T12:56:10.336 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-03-21T12:56:10.336 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-03-21T12:56:10.336 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-03-21T12:56:10.336 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (118 ms) 2024-03-21T12:56:10.336 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (118 ms total) 2024-03-21T12:56:10.336 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-03-21T12:56:10.336 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-03-21T12:56:10.336 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-03-21T12:56:10.336 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (55 ms) 2024-03-21T12:56:10.336 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-03-21T12:56:10.336 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (1 ms) 2024-03-21T12:56:10.336 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (56 ms total) 2024-03-21T12:56:10.336 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-03-21T12:56:10.336 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-03-21T12:56:10.336 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-03-21T12:56:10.336 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (40 ms) 2024-03-21T12:56:10.337 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-03-21T12:56:10.337 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (3 ms) 2024-03-21T12:56:10.337 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (43 ms total) 2024-03-21T12:56:10.337 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-03-21T12:56:10.337 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-03-21T12:56:10.337 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-03-21T12:56:10.337 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (2 ms) 2024-03-21T12:56:10.337 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-03-21T12:56:10.337 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (3 ms) 2024-03-21T12:56:10.337 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (5 ms total) 2024-03-21T12:56:10.337 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-03-21T12:56:10.337 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-03-21T12:56:10.337 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-03-21T12:56:10.338 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1009 ms) 2024-03-21T12:56:10.338 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1009 ms total) 2024-03-21T12:56:10.338 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: 2024-03-21T12:56:10.338 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-03-21T12:56:10.338 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (104655 ms total) 2024-03-21T12:56:10.338 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-03-21T12:56:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:10 smithi155 ceph-mon[33846]: from='client.25987 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81791-36"}]': finished 2024-03-21T12:56:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:10 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-15", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:56:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:10 smithi155 ceph-mon[33846]: osdmap e170: 8 total, 8 up, 8 in 2024-03-21T12:56:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-15"}]: dispatch 2024-03-21T12:56:10.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:10 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-15"}]: dispatch 2024-03-21T12:56:10.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1860962793' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-81661-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:10.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:10 smithi155 ceph-mon[33846]: from='client.26005 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-81661-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:10.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:10.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:10.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:10 smithi043 ceph-mon[30062]: from='client.25987 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81791-36"}]': finished 2024-03-21T12:56:10.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:10 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-15", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:56:10.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:10 smithi043 ceph-mon[30062]: osdmap e170: 8 total, 8 up, 8 in 2024-03-21T12:56:10.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-15"}]: dispatch 2024-03-21T12:56:10.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:10 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-15"}]: dispatch 2024-03-21T12:56:10.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1860962793' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-81661-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:10.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:10 smithi043 ceph-mon[30062]: from='client.26005 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-81661-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:10.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:10.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:10.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:10 smithi043 ceph-mon[41699]: from='client.25987 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi043-81791-36"}]': finished 2024-03-21T12:56:10.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:10 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-15", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:56:10.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:10 smithi043 ceph-mon[41699]: osdmap e170: 8 total, 8 up, 8 in 2024-03-21T12:56:10.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-15"}]: dispatch 2024-03-21T12:56:10.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:10 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-15"}]: dispatch 2024-03-21T12:56:10.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1860962793' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-81661-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:10.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:10 smithi043 ceph-mon[41699]: from='client.26005 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-81661-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:10.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:11 smithi155 ceph-mon[33846]: pgmap v217: 404 pgs: 8 unknown, 32 creating+peering, 364 active+clean; 586 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s wr, 0 op/s 2024-03-21T12:56:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:11 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-15"}]': finished 2024-03-21T12:56:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:11 smithi155 ceph-mon[33846]: from='client.26005 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-81661-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:11 smithi155 ceph-mon[33846]: osdmap e171: 8 total, 8 up, 8 in 2024-03-21T12:56:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:11 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-15", "mode": "writeback"}]: dispatch 2024-03-21T12:56:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:11 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4260480745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:11 smithi155 ceph-mon[33846]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:11 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-15", "mode": "writeback"}]: dispatch 2024-03-21T12:56:11.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:11 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:11 smithi043 ceph-mon[30062]: pgmap v217: 404 pgs: 8 unknown, 32 creating+peering, 364 active+clean; 586 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s wr, 0 op/s 2024-03-21T12:56:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:11 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-15"}]': finished 2024-03-21T12:56:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:11 smithi043 ceph-mon[30062]: from='client.26005 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-81661-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:11 smithi043 ceph-mon[30062]: osdmap e171: 8 total, 8 up, 8 in 2024-03-21T12:56:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:11 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-15", "mode": "writeback"}]: dispatch 2024-03-21T12:56:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:11 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4260480745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:11 smithi043 ceph-mon[30062]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:11 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-15", "mode": "writeback"}]: dispatch 2024-03-21T12:56:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:11 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:11.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:11 smithi043 ceph-mon[41699]: pgmap v217: 404 pgs: 8 unknown, 32 creating+peering, 364 active+clean; 586 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s wr, 0 op/s 2024-03-21T12:56:11.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:11 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-15"}]': finished 2024-03-21T12:56:11.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:11 smithi043 ceph-mon[41699]: from='client.26005 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi043-81661-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:11.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:11 smithi043 ceph-mon[41699]: osdmap e171: 8 total, 8 up, 8 in 2024-03-21T12:56:11.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:11 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-15", "mode": "writeback"}]: dispatch 2024-03-21T12:56:11.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:11 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4260480745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:11.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:11 smithi043 ceph-mon[41699]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:11.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:11 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-15", "mode": "writeback"}]: dispatch 2024-03-21T12:56:11.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:11 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:12 smithi155 ceph-mon[33846]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:12 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:56:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:12 smithi155 ceph-mon[33846]: from='client.25993 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81636-28"}]': finished 2024-03-21T12:56:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:12 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-15", "mode": "writeback"}]': finished 2024-03-21T12:56:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:12 smithi155 ceph-mon[33846]: osdmap e172: 8 total, 8 up, 8 in 2024-03-21T12:56:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:12 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4260480745' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:12 smithi155 ceph-mon[33846]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:12 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:12.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:12 smithi043 ceph-mon[30062]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:12 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:56:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:12 smithi043 ceph-mon[30062]: from='client.25993 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81636-28"}]': finished 2024-03-21T12:56:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:12 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-15", "mode": "writeback"}]': finished 2024-03-21T12:56:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:12 smithi043 ceph-mon[30062]: osdmap e172: 8 total, 8 up, 8 in 2024-03-21T12:56:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:12 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4260480745' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:12 smithi043 ceph-mon[30062]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:12 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:12.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:12 smithi043 ceph-mon[41699]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:12.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:12 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:56:12.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:12 smithi043 ceph-mon[41699]: from='client.25993 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi043-81636-28"}]': finished 2024-03-21T12:56:12.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:12 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-15", "mode": "writeback"}]': finished 2024-03-21T12:56:12.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:12 smithi043 ceph-mon[41699]: osdmap e172: 8 total, 8 up, 8 in 2024-03-21T12:56:12.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:12 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4260480745' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:12.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:12 smithi043 ceph-mon[41699]: from='client.25993 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81636-28"}]: dispatch 2024-03-21T12:56:12.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:12 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:13 smithi155 ceph-mon[33846]: pgmap v220: 428 pgs: 32 unknown, 32 creating+peering, 364 active+clean; 586 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-03-21T12:56:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:13 smithi155 ceph-mon[33846]: from='client.25993 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81636-28"}]': finished 2024-03-21T12:56:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:13 smithi155 ceph-mon[33846]: osdmap e173: 8 total, 8 up, 8 in 2024-03-21T12:56:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:13 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1145852994' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-81661-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:13 smithi155 ceph-mon[33846]: from='client.36014 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-81661-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:13 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3573812120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:13 smithi155 ceph-mon[33846]: from='client.36020 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:13 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3573812120' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:13 smithi155 ceph-mon[33846]: from='client.36020 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:13 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3573812120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-81636-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:13 smithi155 ceph-mon[33846]: from='client.36020 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-81636-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:13 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:13 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:13 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[30062]: pgmap v220: 428 pgs: 32 unknown, 32 creating+peering, 364 active+clean; 586 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-03-21T12:56:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[30062]: from='client.25993 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81636-28"}]': finished 2024-03-21T12:56:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[30062]: osdmap e173: 8 total, 8 up, 8 in 2024-03-21T12:56:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1145852994' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-81661-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[30062]: from='client.36014 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-81661-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3573812120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[30062]: from='client.36020 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3573812120' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[30062]: from='client.36020 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3573812120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-81636-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[30062]: from='client.36020 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-81636-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[41699]: pgmap v220: 428 pgs: 32 unknown, 32 creating+peering, 364 active+clean; 586 KiB data, 574 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-03-21T12:56:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[41699]: from='client.25993 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi043-81636-28"}]': finished 2024-03-21T12:56:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[41699]: osdmap e173: 8 total, 8 up, 8 in 2024-03-21T12:56:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1145852994' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-81661-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[41699]: from='client.36014 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-81661-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3573812120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[41699]: from='client.36020 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3573812120' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[41699]: from='client.36020 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3573812120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-81636-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[41699]: from='client.36020 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-81636-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:13 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:15.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:15 smithi155 ceph-mon[33846]: pgmap v223: 428 pgs: 32 unknown, 396 active+clean; 618 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-03-21T12:56:15.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:15 smithi155 ceph-mon[33846]: from='client.36014 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-81661-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:15.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:15 smithi155 ceph-mon[33846]: from='client.36020 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-81636-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:15.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:15 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:56:15.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:15 smithi155 ceph-mon[33846]: osdmap e174: 8 total, 8 up, 8 in 2024-03-21T12:56:15.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:15 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3573812120' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-81636-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:15.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:15 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-15"}]: dispatch 2024-03-21T12:56:15.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:15 smithi155 ceph-mon[33846]: from='client.36020 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-81636-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:15.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:15 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-15"}]: dispatch 2024-03-21T12:56:15.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:15 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:15.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:15 smithi043 ceph-mon[30062]: pgmap v223: 428 pgs: 32 unknown, 396 active+clean; 618 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-03-21T12:56:15.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:15 smithi043 ceph-mon[30062]: from='client.36014 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-81661-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:15.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:15 smithi043 ceph-mon[30062]: from='client.36020 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-81636-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:15.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:15 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:56:15.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:15 smithi043 ceph-mon[30062]: osdmap e174: 8 total, 8 up, 8 in 2024-03-21T12:56:15.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:15 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3573812120' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-81636-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:15.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:15 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-15"}]: dispatch 2024-03-21T12:56:15.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:15 smithi043 ceph-mon[30062]: from='client.36020 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-81636-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:15.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:15 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-15"}]: dispatch 2024-03-21T12:56:15.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:15 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:15.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:15 smithi043 ceph-mon[41699]: pgmap v223: 428 pgs: 32 unknown, 396 active+clean; 618 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-03-21T12:56:15.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:15 smithi043 ceph-mon[41699]: from='client.36014 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi043-81661-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:15.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:15 smithi043 ceph-mon[41699]: from='client.36020 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi043-81636-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:15.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:15 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:56:15.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:15 smithi043 ceph-mon[41699]: osdmap e174: 8 total, 8 up, 8 in 2024-03-21T12:56:15.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:15 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3573812120' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-81636-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:15.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:15 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-15"}]: dispatch 2024-03-21T12:56:15.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:15 smithi043 ceph-mon[41699]: from='client.36020 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-81636-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:15.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:15 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-15"}]: dispatch 2024-03-21T12:56:15.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:15 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:16.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:16 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:56:16.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:16 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-15"}]': finished 2024-03-21T12:56:16.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:16 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:16.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:16 smithi155 ceph-mon[33846]: osdmap e175: 8 total, 8 up, 8 in 2024-03-21T12:56:16.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:16 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81725-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-21T12:56:16.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:16 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81725-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-21T12:56:16.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:16 smithi155 ceph-mon[33846]: from='client.36020 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-81636-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-81636-29"}]': finished 2024-03-21T12:56:16.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:16 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81725-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-03-21T12:56:16.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:16 smithi155 ceph-mon[33846]: osdmap e176: 8 total, 8 up, 8 in 2024-03-21T12:56:16.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:16 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1860810006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:16.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:16 smithi155 ceph-mon[33846]: from='client.25981 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:16.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:16 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1674882245' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-81661-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:16.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:16 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:56:16.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:16 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-15"}]': finished 2024-03-21T12:56:16.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:16 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:16.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:16 smithi043 ceph-mon[30062]: osdmap e175: 8 total, 8 up, 8 in 2024-03-21T12:56:16.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:16 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81725-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-21T12:56:16.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:16 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81725-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-21T12:56:16.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:16 smithi043 ceph-mon[30062]: from='client.36020 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-81636-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-81636-29"}]': finished 2024-03-21T12:56:16.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:16 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81725-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-03-21T12:56:16.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:16 smithi043 ceph-mon[30062]: osdmap e176: 8 total, 8 up, 8 in 2024-03-21T12:56:16.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:16 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1860810006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:16.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:16 smithi043 ceph-mon[30062]: from='client.25981 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:16.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:16 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1674882245' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-81661-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:16 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:56:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:16 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-15"}]': finished 2024-03-21T12:56:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:16 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:16 smithi043 ceph-mon[41699]: osdmap e175: 8 total, 8 up, 8 in 2024-03-21T12:56:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:16 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81725-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-21T12:56:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:16 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81725-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-21T12:56:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:16 smithi043 ceph-mon[41699]: from='client.36020 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi043-81636-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi043-81636-29"}]': finished 2024-03-21T12:56:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:16 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi043-81725-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-03-21T12:56:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:16 smithi043 ceph-mon[41699]: osdmap e176: 8 total, 8 up, 8 in 2024-03-21T12:56:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:16 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1860810006' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:16 smithi043 ceph-mon[41699]: from='client.25981 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:16 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1674882245' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-81661-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:17.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:17 smithi155 ceph-mon[33846]: pgmap v226: 396 pgs: 396 active+clean; 618 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-03-21T12:56:17.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:17 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:17.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:17 smithi155 ceph-mon[33846]: from='client.36026 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-81661-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:17.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:17 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:17 smithi155 ceph-mon[33846]: from='client.25981 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]': finished 2024-03-21T12:56:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:17 smithi155 ceph-mon[33846]: from='client.36026 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-81661-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:17 smithi155 ceph-mon[33846]: osdmap e177: 8 total, 8 up, 8 in 2024-03-21T12:56:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:17 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1860810006' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:17 smithi155 ceph-mon[33846]: from='client.25981 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:17 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:17 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:17 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:17 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:17 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-21T12:56:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:17 smithi155 ceph-mon[33846]: from='client.24697 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-21T12:56:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[30062]: pgmap v226: 396 pgs: 396 active+clean; 618 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-03-21T12:56:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[30062]: from='client.36026 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-81661-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[30062]: from='client.25981 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]': finished 2024-03-21T12:56:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[30062]: from='client.36026 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-81661-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[30062]: osdmap e177: 8 total, 8 up, 8 in 2024-03-21T12:56:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1860810006' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:17.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[30062]: from='client.25981 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:17.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:17.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:17.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:17.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:17.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-21T12:56:17.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[30062]: from='client.24697 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-21T12:56:17.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[41699]: pgmap v226: 396 pgs: 396 active+clean; 618 KiB data, 579 MiB used, 715 GiB / 715 GiB avail; 7.2 KiB/s rd, 10 KiB/s wr, 4 op/s 2024-03-21T12:56:17.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:17.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[41699]: from='client.36026 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-81661-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:17.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:17.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[41699]: from='client.25981 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]': finished 2024-03-21T12:56:17.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[41699]: from='client.36026 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi043-81661-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:17.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[41699]: osdmap e177: 8 total, 8 up, 8 in 2024-03-21T12:56:17.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1860810006' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:17.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[41699]: from='client.25981 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]: dispatch 2024-03-21T12:56:17.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:17.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:17.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:17.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:17.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-21T12:56:17.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:17 smithi043 ceph-mon[41699]: from='client.24697 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-21T12:56:18.339 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-03-21T12:56:18.339 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-03-21T12:56:18.339 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-03-21T12:56:18.339 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-03-21T12:56:18.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-03-21T12:56:18.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (10919 ms) 2024-03-21T12:56:18.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-03-21T12:56:18.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (22901 ms) 2024-03-21T12:56:18.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-03-21T12:56:18.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (6245 ms) 2024-03-21T12:56:18.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-03-21T12:56:18.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (2629 ms) 2024-03-21T12:56:18.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-03-21T12:56:18.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (4025 ms) 2024-03-21T12:56:18.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-03-21T12:56:18.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (2988 ms) 2024-03-21T12:56:18.340 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-03-21T12:56:18.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (2994 ms) 2024-03-21T12:56:18.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-03-21T12:56:18.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (3016 ms) 2024-03-21T12:56:18.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-03-21T12:56:18.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (2993 ms) 2024-03-21T12:56:18.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-03-21T12:56:18.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (2997 ms) 2024-03-21T12:56:18.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-03-21T12:56:18.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (3005 ms) 2024-03-21T12:56:18.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-03-21T12:56:18.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (2995 ms) 2024-03-21T12:56:18.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-03-21T12:56:18.341 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3017 ms) 2024-03-21T12:56:18.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-03-21T12:56:18.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3017 ms) 2024-03-21T12:56:18.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-03-21T12:56:18.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3009 ms) 2024-03-21T12:56:18.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-03-21T12:56:18.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3021 ms) 2024-03-21T12:56:18.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-03-21T12:56:18.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (3013 ms) 2024-03-21T12:56:18.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-03-21T12:56:18.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (2997 ms) 2024-03-21T12:56:18.342 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-03-21T12:56:18.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3005 ms) 2024-03-21T12:56:18.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-03-21T12:56:18.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (2924 ms) 2024-03-21T12:56:18.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-03-21T12:56:18.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (3008 ms) 2024-03-21T12:56:18.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-03-21T12:56:18.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3014 ms) 2024-03-21T12:56:18.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-03-21T12:56:18.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (3018 ms) 2024-03-21T12:56:18.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-03-21T12:56:18.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (2978 ms) 2024-03-21T12:56:18.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-03-21T12:56:18.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (2991 ms) 2024-03-21T12:56:18.343 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-03-21T12:56:18.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (3259 ms) 2024-03-21T12:56:18.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-03-21T12:56:18.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAinapshotsECPP.SnapRemovePP (2004 ms) 2024-03-21T12:56:18.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-03-21T12:56:18.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2010 ms) 2024-03-21T12:56:18.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-03-21T12:56:18.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (1919 ms) 2024-03-21T12:56:18.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (8939 ms total) 2024-03-21T12:56:18.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: 2024-03-21T12:56:18.344 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-03-21T12:56:18.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-03-21T12:56:18.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4008 ms) 2024-03-21T12:56:18.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-03-21T12:56:18.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3987 ms) 2024-03-21T12:56:18.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-03-21T12:56:18.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (3997 ms) 2024-03-21T12:56:18.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (11992 ms total) 2024-03-21T12:56:18.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: 2024-03-21T12:56:18.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-03-21T12:56:18.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [==========] 21 tests from 5 test suites ran. (112616 ms total) 2024-03-21T12:56:18.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-03-21T12:56:18.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ SKIPPED ] 1 test, listed below: 2024-03-21T12:56:18.345 INFO:tasks.workunit.client.0.smithi043.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-03-21T12:56:18.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:56:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:56:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:56:18.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:56:18 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:56:19.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:19 smithi155 ceph-mon[33846]: pgmap v229: 428 pgs: 72 unknown, 356 active+clean; 586 KiB data, 579 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:56:19.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:19 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:56:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:19 smithi155 ceph-mon[33846]: from='client.25981 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]': finished 2024-03-21T12:56:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:19 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:19 smithi155 ceph-mon[33846]: from='client.24697 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pgp_num","val":"11"}]': finished 2024-03-21T12:56:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:19 smithi155 ceph-mon[33846]: osdmap e178: 8 total, 8 up, 8 in 2024-03-21T12:56:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:19 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:19 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3573812120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:19 smithi155 ceph-mon[33846]: from='client.36020 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:19 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:19 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:19 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:56:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:19 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:56:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[30062]: pgmap v229: 428 pgs: 72 unknown, 356 active+clean; 586 KiB data, 579 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:56:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:56:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[30062]: from='client.25981 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]': finished 2024-03-21T12:56:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[30062]: from='client.24697 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pgp_num","val":"11"}]': finished 2024-03-21T12:56:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[30062]: osdmap e178: 8 total, 8 up, 8 in 2024-03-21T12:56:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3573812120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[30062]: from='client.36020 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:56:19.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:56:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[41699]: pgmap v229: 428 pgs: 72 unknown, 356 active+clean; 586 KiB data, 579 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:56:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:56:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[41699]: from='client.25981 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi043-82022-21"}]': finished 2024-03-21T12:56:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[41699]: from='client.24697 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi043-81725-1","var":"pgp_num","val":"11"}]': finished 2024-03-21T12:56:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[41699]: osdmap e178: 8 total, 8 up, 8 in 2024-03-21T12:56:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1225806232' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-21T12:56:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3573812120' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[41699]: from='client.36020 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:56:19.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:19 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:56:20.361 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:20 smithi155 ceph-mon[33846]: from='client.36020 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-81636-29"}]': finished 2024-03-21T12:56:20.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:20 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-17", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:56:20.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:20 smithi155 ceph-mon[33846]: osdmap e179: 8 total, 8 up, 8 in 2024-03-21T12:56:20.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:20 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3573812120' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:20.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:20 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-17"}]: dispatch 2024-03-21T12:56:20.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:20 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/617715945' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-81661-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:20.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:20 smithi155 ceph-mon[33846]: from='client.26029 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-81661-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:20.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:20 smithi155 ceph-mon[33846]: from='client.36020 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:20.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:20 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-17"}]: dispatch 2024-03-21T12:56:20.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:20 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1868413591' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:20.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:20 smithi155 ceph-mon[33846]: from='client.36038 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:20.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:20 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1868413591' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:20.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:20 smithi155 ceph-mon[33846]: from='client.36038 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:20.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:20 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1868413591' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-81725-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:20.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:20 smithi155 ceph-mon[33846]: from='client.36038 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-81725-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:20.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:20 smithi155 ceph-mon[33846]: from='client.26029 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-81661-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[30062]: from='client.36020 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-81636-29"}]': finished 2024-03-21T12:56:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-17", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:56:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[30062]: osdmap e179: 8 total, 8 up, 8 in 2024-03-21T12:56:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3573812120' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-17"}]: dispatch 2024-03-21T12:56:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/617715945' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-81661-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[30062]: from='client.26029 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-81661-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[30062]: from='client.36020 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-17"}]: dispatch 2024-03-21T12:56:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1868413591' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[30062]: from='client.36038 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:20.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1868413591' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:20.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[30062]: from='client.36038 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:20.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1868413591' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-81725-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:20.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[30062]: from='client.36038 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-81725-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:20.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[30062]: from='client.26029 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-81661-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:20.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[41699]: from='client.36020 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi043-81636-29"}]': finished 2024-03-21T12:56:20.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-17", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:56:20.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[41699]: osdmap e179: 8 total, 8 up, 8 in 2024-03-21T12:56:20.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3573812120' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:20.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-17"}]: dispatch 2024-03-21T12:56:20.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/617715945' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-81661-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:20.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[41699]: from='client.26029 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-81661-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:20.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[41699]: from='client.36020 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-81636-29"}]: dispatch 2024-03-21T12:56:20.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-17"}]: dispatch 2024-03-21T12:56:20.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1868413591' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:20.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[41699]: from='client.36038 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:20.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1868413591' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:20.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[41699]: from='client.36038 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:20.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1868413591' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-81725-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:20.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[41699]: from='client.36038 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-81725-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:20.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:20 smithi043 ceph-mon[41699]: from='client.26029 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi043-81661-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:21.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:21 smithi155 ceph-mon[33846]: pgmap v232: 388 pgs: 32 unknown, 32 creating+peering, 324 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:56:21.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:21 smithi155 ceph-mon[33846]: from='client.36020 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-81636-29"}]': finished 2024-03-21T12:56:21.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:21 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-17"}]': finished 2024-03-21T12:56:21.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:21 smithi155 ceph-mon[33846]: from='client.36038 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-81725-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:21.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:21 smithi155 ceph-mon[33846]: osdmap e180: 8 total, 8 up, 8 in 2024-03-21T12:56:21.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:21 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-17", "mode": "writeback"}]: dispatch 2024-03-21T12:56:21.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:21 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1868413591' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-81725-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:21.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:21 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-17", "mode": "writeback"}]: dispatch 2024-03-21T12:56:21.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:21 smithi155 ceph-mon[33846]: from='client.36038 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-81725-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:21.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:21 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4232336200' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:21.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:21 smithi155 ceph-mon[33846]: from='client.36044 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:21.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:21 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4232336200' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:21.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:21 smithi155 ceph-mon[33846]: from='client.36044 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:21.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:21 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4232336200' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-81636-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:21.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:21 smithi155 ceph-mon[33846]: from='client.36044 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-81636-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:21.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:21 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:56:21.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:21 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-17", "mode": "writeback"}]': finished 2024-03-21T12:56:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[30062]: pgmap v232: 388 pgs: 32 unknown, 32 creating+peering, 324 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:56:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[30062]: from='client.36020 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-81636-29"}]': finished 2024-03-21T12:56:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-17"}]': finished 2024-03-21T12:56:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[30062]: from='client.36038 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-81725-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[30062]: osdmap e180: 8 total, 8 up, 8 in 2024-03-21T12:56:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-17", "mode": "writeback"}]: dispatch 2024-03-21T12:56:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1868413591' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-81725-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-17", "mode": "writeback"}]: dispatch 2024-03-21T12:56:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[30062]: from='client.36038 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-81725-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4232336200' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[30062]: from='client.36044 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4232336200' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[30062]: from='client.36044 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:21.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4232336200' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-81636-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:21.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[30062]: from='client.36044 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-81636-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:21.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:56:21.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-17", "mode": "writeback"}]': finished 2024-03-21T12:56:21.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[41699]: pgmap v232: 388 pgs: 32 unknown, 32 creating+peering, 324 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:56:21.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[41699]: from='client.36020 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi043-81636-29"}]': finished 2024-03-21T12:56:21.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-17"}]': finished 2024-03-21T12:56:21.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[41699]: from='client.36038 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi043-81725-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:21.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[41699]: osdmap e180: 8 total, 8 up, 8 in 2024-03-21T12:56:21.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-17", "mode": "writeback"}]: dispatch 2024-03-21T12:56:21.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1868413591' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-81725-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:21.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-17", "mode": "writeback"}]: dispatch 2024-03-21T12:56:21.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[41699]: from='client.36038 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-81725-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:21.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4232336200' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:21.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[41699]: from='client.36044 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:21.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4232336200' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:21.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[41699]: from='client.36044 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:21.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4232336200' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-81636-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:21.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[41699]: from='client.36044 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-81636-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:21.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:56:21.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:21 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-17", "mode": "writeback"}]': finished 2024-03-21T12:56:22.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:22 smithi155 ceph-mon[33846]: from='client.36044 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-81636-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:22.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:22 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4232336200' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-81636-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:22.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:22 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:22.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:22 smithi155 ceph-mon[33846]: osdmap e181: 8 total, 8 up, 8 in 2024-03-21T12:56:22.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:22 smithi155 ceph-mon[33846]: from='client.36044 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-81636-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:22.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:22 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:22.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:22 smithi155 ceph-mon[33846]: from='client.36038 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-81725-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-81725-2"}]': finished 2024-03-21T12:56:22.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:22 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:56:22.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:22 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-17"}]: dispatch 2024-03-21T12:56:22.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:22 smithi155 ceph-mon[33846]: osdmap e182: 8 total, 8 up, 8 in 2024-03-21T12:56:22.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:22 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-17"}]: dispatch 2024-03-21T12:56:22.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:22 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3978466839' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-81661-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:22 smithi043 ceph-mon[30062]: from='client.36044 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-81636-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:22 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4232336200' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-81636-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:22 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:22 smithi043 ceph-mon[30062]: osdmap e181: 8 total, 8 up, 8 in 2024-03-21T12:56:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:22 smithi043 ceph-mon[30062]: from='client.36044 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-81636-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:22 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:22 smithi043 ceph-mon[30062]: from='client.36038 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-81725-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-81725-2"}]': finished 2024-03-21T12:56:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:22 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:56:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:22 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-17"}]: dispatch 2024-03-21T12:56:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:22 smithi043 ceph-mon[30062]: osdmap e182: 8 total, 8 up, 8 in 2024-03-21T12:56:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:22 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-17"}]: dispatch 2024-03-21T12:56:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:22 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3978466839' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-81661-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:22.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:22 smithi043 ceph-mon[41699]: from='client.36044 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi043-81636-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:22.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:22 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4232336200' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-81636-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:22.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:22 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:22.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:22 smithi043 ceph-mon[41699]: osdmap e181: 8 total, 8 up, 8 in 2024-03-21T12:56:22.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:22 smithi043 ceph-mon[41699]: from='client.36044 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-81636-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:22.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:22 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:22.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:22 smithi043 ceph-mon[41699]: from='client.36038 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi043-81725-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi043-81725-2"}]': finished 2024-03-21T12:56:22.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:22 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:56:22.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:22 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-17"}]: dispatch 2024-03-21T12:56:22.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:22 smithi043 ceph-mon[41699]: osdmap e182: 8 total, 8 up, 8 in 2024-03-21T12:56:22.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:22 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-17"}]: dispatch 2024-03-21T12:56:22.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:22 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3978466839' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-81661-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:23.398 INFO:tasks.workunit.client.0.smithi043.stdout:pi_list: : entry=10 expected=10 2024-03-21T12:56:23.398 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:f905c69b:::2:head 2024-03-21T12:56:23.398 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=13:f905c69b:::2:head expected=13:f905c69b:::2:head 2024-03-21T12:56:23.398 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:f905c69b:::2:head -> 2 2024-03-21T12:56:23.398 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=2 expected=2 2024-03-21T12:56:23.398 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:cfc208b3:::3:head 2024-03-21T12:56:23.398 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=13:cfc208b3:::3:head expected=13:cfc208b3:::3:head 2024-03-21T12:56:23.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:cfc208b3:::3:head -> 3 2024-03-21T12:56:23.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=3 expected=3 2024-03-21T12:56:23.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:c4fdafeb:::6:head 2024-03-21T12:56:23.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=13:c4fdafeb:::6:head expected=13:c4fdafeb:::6:head 2024-03-21T12:56:23.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:c4fdafeb:::6:head -> 6 2024-03-21T12:56:23.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=6 expected=6 2024-03-21T12:56:23.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:bd63b0f1:::8:head 2024-03-21T12:56:23.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=13:bd63b0f1:::8:head expected=13:bd63b0f1:::8:head 2024-03-21T12:56:23.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:bd63b0f1:::8:head -> 8 2024-03-21T12:56:23.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=8 expected=8 2024-03-21T12:56:23.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:b29083e3:::5:head 2024-03-21T12:56:23.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=13:b29083e3:::5:head expected=13:b29083e3:::5:head 2024-03-21T12:56:23.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:b29083e3:::5:head -> 5 2024-03-21T12:56:23.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=5 expected=5 2024-03-21T12:56:23.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:e960b815:::9:head 2024-03-21T12:56:23.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=13:e960b815:::9:head expected=13:e960b815:::9:head 2024-03-21T12:56:23.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:e960b815:::9:head -> 9 2024-03-21T12:56:23.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=9 expected=9 2024-03-21T12:56:23.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:de5d7c5f:::12:head 2024-03-21T12:56:23.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=13:de5d7c5f:::12:head expected=13:de5d7c5f:::12:head 2024-03-21T12:56:23.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:de5d7c5f:::12:head -> 12 2024-03-21T12:56:23.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=12 expected=12 2024-03-21T12:56:23.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : seek to 13:d83876eb:::4:head 2024-03-21T12:56:23.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : cursor()=13:d83876eb:::4:head expected=13:d83876eb:::4:head 2024-03-21T12:56:23.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: > 13:d83876eb:::4:head -> 4 2024-03-21T12:56:23.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: : entry=4 expected=4 2024-03-21T12:56:23.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (264 ms) 2024-03-21T12:56:23.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-03-21T12:56:23.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (58626 ms) 2024-03-21T12:56:23.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-03-21T12:56:23.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: split 0/5 -> MIN 13:33333333::::head 2024-03-21T12:56:23.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: split 1/5 -> 13:33333333::::head 13:66666666::::head 2024-03-21T12:56:23.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: split 2/5 -> 13:66666666::::head 13:99999999::::head 2024-03-21T12:56:23.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: split 3/5 -> 13:99999999::::head 13:cccccccc::::head 2024-03-21T12:56:23.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: split 4/5 -> 13:cccccccc::::head MAX 2024-03-21T12:56:23.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (43025 ms) 2024-03-21T12:56:23.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] 7 tests from LibRadosList (102532 ms total) 2024-03-21T12:56:23.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 2024-03-21T12:56:23.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-03-21T12:56:23.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-03-21T12:56:23.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1043 ms) 2024-03-21T12:56:23.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-03-21T12:56:23.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset foo1,foo2,foo3 2024-03-21T12:56:23.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo1 2024-03-21T12:56:23.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo2 2024-03-21T12:56:23.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo3 2024-03-21T12:56:23.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset foo1,foo4,foo5 2024-03-21T12:56:23.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo4 2024-03-21T12:56:23.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo5 2024-03-21T12:56:23.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo1 2024-03-21T12:56:23.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset foo6,foo7 2024-03-21T12:56:23.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo7 2024-03-21T12:56:23.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: foo6 2024-03-21T12:56:23.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-03-21T12:56:23.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns1:foo4 2024-03-21T12:56:23.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns1:foo5 2024-03-21T12:56:23.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns2:foo7 2024-03-21T12:56:23.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns2:foo6 2024-03-21T12:56:23.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: ns1:foo1 2024-03-21T12:56:23.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: :foo1 2024-03-21T12:56:23.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: :foo2 2024-03-21T12:56:23.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: :foo3 2024-03-21T12:56:23.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (31 ms) 2024-03-21T12:56:23.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:23 smithi155 ceph-mon[33846]: pgmap v235: 356 pgs: 32 creating+peering, 324 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:56:23.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:23 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:23.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:23 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:56:23.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:23 smithi155 ceph-mon[33846]: from='client.36044 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-81636-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-81636-30"}]': finished 2024-03-21T12:56:23.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:23 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-17"}]': finished 2024-03-21T12:56:23.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:23 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3978466839' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-81661-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:23.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:23 smithi155 ceph-mon[33846]: osdmap e183: 8 total, 8 up, 8 in 2024-03-21T12:56:23.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:23 smithi043 ceph-mon[30062]: pgmap v235: 356 pgs: 32 creating+peering, 324 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:56:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:23 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:23 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:56:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:23 smithi043 ceph-mon[30062]: from='client.36044 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-81636-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-81636-30"}]': finished 2024-03-21T12:56:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:23 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-17"}]': finished 2024-03-21T12:56:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:23 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3978466839' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-81661-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:23 smithi043 ceph-mon[30062]: osdmap e183: 8 total, 8 up, 8 in 2024-03-21T12:56:23.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:23 smithi043 ceph-mon[41699]: pgmap v235: 356 pgs: 32 creating+peering, 324 active+clean; 584 KiB data, 580 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:56:23.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:23 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:23.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:23 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:56:23.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:23 smithi043 ceph-mon[41699]: from='client.36044 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi043-81636-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi043-81636-30"}]': finished 2024-03-21T12:56:23.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:23 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-17"}]': finished 2024-03-21T12:56:23.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:23 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3978466839' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi043-81661-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:23.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:23 smithi043 ceph-mon[41699]: osdmap e183: 8 total, 8 up, 8 in 2024-03-21T12:56:25.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:25 smithi155 ceph-mon[33846]: pgmap v238: 404 pgs: 8 unknown, 40 creating+peering, 356 active+clean; 584 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-21T12:56:25.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:25 smithi155 ceph-mon[33846]: osdmap e184: 8 total, 8 up, 8 in 2024-03-21T12:56:25.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:25 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1868413591' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:25.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:25 smithi155 ceph-mon[33846]: from='client.36038 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:25.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:25 smithi043 ceph-mon[30062]: pgmap v238: 404 pgs: 8 unknown, 40 creating+peering, 356 active+clean; 584 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-21T12:56:25.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:25 smithi043 ceph-mon[30062]: osdmap e184: 8 total, 8 up, 8 in 2024-03-21T12:56:25.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:25 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1868413591' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:25.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:25 smithi043 ceph-mon[30062]: from='client.36038 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:25.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:25 smithi043 ceph-mon[41699]: pgmap v238: 404 pgs: 8 unknown, 40 creating+peering, 356 active+clean; 584 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-21T12:56:25.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:25 smithi043 ceph-mon[41699]: osdmap e184: 8 total, 8 up, 8 in 2024-03-21T12:56:25.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:25 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1868413591' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:25.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:25 smithi043 ceph-mon[41699]: from='client.36038 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:26.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:26 smithi155 ceph-mon[33846]: from='client.36038 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81725-2"}]': finished 2024-03-21T12:56:26.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:26 smithi155 ceph-mon[33846]: osdmap e185: 8 total, 8 up, 8 in 2024-03-21T12:56:26.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:26 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4232336200' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:26.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:26 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:26.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:26 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1868413591' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:26.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:26 smithi155 ceph-mon[33846]: from='client.36044 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:26.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:26 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/790306891' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-81661-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:26.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:26 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:26.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:26 smithi155 ceph-mon[33846]: from='client.36038 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:26.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:26 smithi155 ceph-mon[33846]: from='client.36053 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-81661-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:26.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:26 smithi043 ceph-mon[30062]: from='client.36038 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81725-2"}]': finished 2024-03-21T12:56:26.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:26 smithi043 ceph-mon[30062]: osdmap e185: 8 total, 8 up, 8 in 2024-03-21T12:56:26.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:26 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4232336200' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:26.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:26 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:26.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:26 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1868413591' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:26.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:26 smithi043 ceph-mon[30062]: from='client.36044 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:26.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:26 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/790306891' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-81661-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:26.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:26 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:26.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:26 smithi043 ceph-mon[30062]: from='client.36038 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:26.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:26 smithi043 ceph-mon[30062]: from='client.36053 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-81661-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:26.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:26 smithi043 ceph-mon[41699]: from='client.36038 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi043-81725-2"}]': finished 2024-03-21T12:56:26.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:26 smithi043 ceph-mon[41699]: osdmap e185: 8 total, 8 up, 8 in 2024-03-21T12:56:26.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:26 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4232336200' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:26.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:26 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:26.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:26 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1868413591' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:26.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:26 smithi043 ceph-mon[41699]: from='client.36044 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:26.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:26 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/790306891' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-81661-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:26.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:26 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:26.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:26 smithi043 ceph-mon[41699]: from='client.36038 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81725-2"}]: dispatch 2024-03-21T12:56:26.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:26 smithi043 ceph-mon[41699]: from='client.36053 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-81661-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:27.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:27 smithi155 ceph-mon[33846]: pgmap v241: 388 pgs: 64 unknown, 324 active+clean; 584 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T12:56:27.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:27 smithi155 ceph-mon[33846]: from='client.36044 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-81636-30"}]': finished 2024-03-21T12:56:27.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:27 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:27.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:27 smithi155 ceph-mon[33846]: from='client.36038 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81725-2"}]': finished 2024-03-21T12:56:27.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:27 smithi155 ceph-mon[33846]: from='client.36053 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-81661-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:27.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:27 smithi155 ceph-mon[33846]: osdmap e186: 8 total, 8 up, 8 in 2024-03-21T12:56:27.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:27 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4232336200' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:27.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:27 smithi155 ceph-mon[33846]: from='client.36044 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:27.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:27 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:27.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:27 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:27.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:27 smithi043 ceph-mon[30062]: pgmap v241: 388 pgs: 64 unknown, 324 active+clean; 584 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T12:56:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:27 smithi043 ceph-mon[30062]: from='client.36044 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-81636-30"}]': finished 2024-03-21T12:56:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:27 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:27 smithi043 ceph-mon[30062]: from='client.36038 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81725-2"}]': finished 2024-03-21T12:56:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:27 smithi043 ceph-mon[30062]: from='client.36053 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-81661-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:27 smithi043 ceph-mon[30062]: osdmap e186: 8 total, 8 up, 8 in 2024-03-21T12:56:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:27 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4232336200' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:27 smithi043 ceph-mon[30062]: from='client.36044 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:27 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:27 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:27.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:27 smithi043 ceph-mon[41699]: pgmap v241: 388 pgs: 64 unknown, 324 active+clean; 584 KiB data, 585 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T12:56:27.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:27 smithi043 ceph-mon[41699]: from='client.36044 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi043-81636-30"}]': finished 2024-03-21T12:56:27.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:27 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:27.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:27 smithi043 ceph-mon[41699]: from='client.36038 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi043-81725-2"}]': finished 2024-03-21T12:56:27.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:27 smithi043 ceph-mon[41699]: from='client.36053 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi043-81661-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:27.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:27 smithi043 ceph-mon[41699]: osdmap e186: 8 total, 8 up, 8 in 2024-03-21T12:56:27.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:27 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4232336200' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:27.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:27 smithi043 ceph-mon[41699]: from='client.36044 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-81636-30"}]: dispatch 2024-03-21T12:56:27.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:27 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:27.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:27 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:28.349 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:56:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:56:28.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:56:28 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:56:28.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:28 smithi155 ceph-mon[33846]: from='client.36044 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-81636-30"}]': finished 2024-03-21T12:56:28.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:28 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-19", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:56:28.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:28 smithi155 ceph-mon[33846]: osdmap e187: 8 total, 8 up, 8 in 2024-03-21T12:56:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1243035879' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81725-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-19"}]: dispatch 2024-03-21T12:56:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:28 smithi155 ceph-mon[33846]: from='client.36059 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81725-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:28 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-19"}]: dispatch 2024-03-21T12:56:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3369657660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:28 smithi155 ceph-mon[33846]: from='client.26065 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3369657660' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:28 smithi155 ceph-mon[33846]: from='client.26065 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3369657660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-81636-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:28 smithi155 ceph-mon[33846]: from='client.26065 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-81636-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:28.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[30062]: from='client.36044 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-81636-30"}]': finished 2024-03-21T12:56:28.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-19", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:56:28.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[30062]: osdmap e187: 8 total, 8 up, 8 in 2024-03-21T12:56:28.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1243035879' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81725-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:28.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-19"}]: dispatch 2024-03-21T12:56:28.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[30062]: from='client.36059 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81725-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:28.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-19"}]: dispatch 2024-03-21T12:56:28.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3369657660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:28.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[30062]: from='client.26065 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:28.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3369657660' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:28.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[30062]: from='client.26065 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:28.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3369657660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-81636-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:28.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[30062]: from='client.26065 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-81636-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:28.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[41699]: from='client.36044 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi043-81636-30"}]': finished 2024-03-21T12:56:28.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-19", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:56:28.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[41699]: osdmap e187: 8 total, 8 up, 8 in 2024-03-21T12:56:28.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1243035879' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81725-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:28.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-19"}]: dispatch 2024-03-21T12:56:28.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[41699]: from='client.36059 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81725-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:28.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-19"}]: dispatch 2024-03-21T12:56:28.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3369657660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:28.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[41699]: from='client.26065 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:28.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3369657660' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:28.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[41699]: from='client.26065 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:28.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3369657660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-81636-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:28.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:28 smithi043 ceph-mon[41699]: from='client.26065 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-81636-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:29.363 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2024-03-21T12:56:29.364 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 1 0 2024-03-21T12:56:29.364 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 10 0 2024-03-21T12:56:29.364 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 13 0 2024-03-21T12:56:29.364 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 7 0 2024-03-21T12:56:29.364 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 14 0 2024-03-21T12:56:29.364 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 0 0 2024-03-21T12:56:29.364 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 15 0 2024-03-21T12:56:29.364 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 11 0 2024-03-21T12:56:29.364 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 5 0 2024-03-21T12:56:29.364 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 8 0 2024-03-21T12:56:29.364 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 6 0 2024-03-21T12:56:29.364 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 3 0 2024-03-21T12:56:29.365 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 4 0 2024-03-21T12:56:29.365 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 12 0 2024-03-21T12:56:29.365 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 9 0 2024-03-21T12:56:29.365 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 2 0 2024-03-21T12:56:29.365 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-03-21T12:56:29.365 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (42 ms) 2024-03-21T12:56:29.365 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] 3 tests from LibRadosListEC (1116 ms total) 2024-03-21T12:56:29.365 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 2024-03-21T12:56:29.365 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-03-21T12:56:29.365 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-03-21T12:56:29.365 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (3011 ms) 2024-03-21T12:56:29.365 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] 1 test from LibRadosListNP (3011 ms total) 2024-03-21T12:56:29.365 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: 2024-03-21T12:56:29.365 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [----------] Global test environment tear-down 2024-03-21T12:56:29.365 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [==========] 11 tests from 3 test suites ran. (123716 ms total) 2024-03-21T12:56:29.365 INFO:tasks.workunit.client.0.smithi043.stdout: api_list: [ PASSED ] 11 tests. 2024-03-21T12:56:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:29 smithi155 ceph-mon[33846]: pgmap v244: 388 pgs: 64 unknown, 324 active+clean; 584 KiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:56:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:29 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:56:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:29 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:29 smithi155 ceph-mon[33846]: from='client.36059 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81725-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:29 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-19"}]': finished 2024-03-21T12:56:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:29 smithi155 ceph-mon[33846]: from='client.26065 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-81636-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:29.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:29 smithi155 ceph-mon[33846]: osdmap e188: 8 total, 8 up, 8 in 2024-03-21T12:56:29.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:29 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-19", "mode": "writeback"}]: dispatch 2024-03-21T12:56:29.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:29 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3369657660' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-81636-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:29.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:29 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-19", "mode": "writeback"}]: dispatch 2024-03-21T12:56:29.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:29 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/580096518' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-81661-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:29.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:29 smithi155 ceph-mon[33846]: from='client.26065 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-81636-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:29.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:29 smithi155 ceph-mon[33846]: from='client.26062 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-81661-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:29.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:29 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1243035879' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-81725-3","pool2":"test-rados-api-smithi043-81725-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-21T12:56:29.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:29 smithi155 ceph-mon[33846]: from='client.36059 ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-81725-3","pool2":"test-rados-api-smithi043-81725-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-21T12:56:29.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[30062]: pgmap v244: 388 pgs: 64 unknown, 324 active+clean; 584 KiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:56:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:56:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[30062]: from='client.36059 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81725-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-19"}]': finished 2024-03-21T12:56:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[30062]: from='client.26065 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-81636-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[30062]: osdmap e188: 8 total, 8 up, 8 in 2024-03-21T12:56:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-19", "mode": "writeback"}]: dispatch 2024-03-21T12:56:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3369657660' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-81636-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-19", "mode": "writeback"}]: dispatch 2024-03-21T12:56:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/580096518' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-81661-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[30062]: from='client.26065 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-81636-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[30062]: from='client.26062 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-81661-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1243035879' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-81725-3","pool2":"test-rados-api-smithi043-81725-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-21T12:56:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[30062]: from='client.36059 ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-81725-3","pool2":"test-rados-api-smithi043-81725-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-21T12:56:29.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[41699]: pgmap v244: 388 pgs: 64 unknown, 324 active+clean; 584 KiB data, 585 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:56:29.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:56:29.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:29.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[41699]: from='client.36059 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81725-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:29.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-19"}]': finished 2024-03-21T12:56:29.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[41699]: from='client.26065 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi043-81636-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:29.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[41699]: osdmap e188: 8 total, 8 up, 8 in 2024-03-21T12:56:29.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-19", "mode": "writeback"}]: dispatch 2024-03-21T12:56:29.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3369657660' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-81636-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:29.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-19", "mode": "writeback"}]: dispatch 2024-03-21T12:56:29.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/580096518' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-81661-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:29.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[41699]: from='client.26065 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-81636-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:29.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[41699]: from='client.26062 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-81661-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:29.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1243035879' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-81725-3","pool2":"test-rados-api-smithi043-81725-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-21T12:56:29.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:29 smithi043 ceph-mon[41699]: from='client.36059 ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-81725-3","pool2":"test-rados-api-smithi043-81725-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-21T12:56:30.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:56:30.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-19", "mode": "writeback"}]': finished 2024-03-21T12:56:30.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: from='client.26062 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-81661-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: from='client.36059 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-81725-3","pool2":"test-rados-api-smithi043-81725-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-03-21T12:56:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: osdmap e189: 8 total, 8 up, 8 in 2024-03-21T12:56:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/580096518' entity='client.admin' cmd=[{ 2024-03-21T12:56:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: "prefix": "osd pool set", 2024-03-21T12:56:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: "pool": "PoolEIOFlag_smithi043-81661-33", 2024-03-21T12:56:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: "var": "eio", 2024-03-21T12:56:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: "val": "true" 2024-03-21T12:56:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: }]: dispatch 2024-03-21T12:56:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: from='client.26062 ' entity='client.admin' cmd=[{ 2024-03-21T12:56:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: "prefix": "osd pool set", 2024-03-21T12:56:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: "pool": "PoolEIOFlag_smithi043-81661-33", 2024-03-21T12:56:30.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: "var": "eio", 2024-03-21T12:56:30.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: "val": "true" 2024-03-21T12:56:30.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: }]: dispatch 2024-03-21T12:56:30.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: from='client.26065 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-81636-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-81636-31"}]': finished 2024-03-21T12:56:30.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:56:30.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: from='client.26062 ' entity='client.admin' cmd='[{ 2024-03-21T12:56:30.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: "prefix": "osd pool set", 2024-03-21T12:56:30.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: "pool": "PoolEIOFlag_smithi043-81661-33", 2024-03-21T12:56:30.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: "var": "eio", 2024-03-21T12:56:30.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: "val": "true" 2024-03-21T12:56:30.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: }]': finished 2024-03-21T12:56:30.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: osdmap e190: 8 total, 8 up, 8 in 2024-03-21T12:56:30.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-19"}]: dispatch 2024-03-21T12:56:30.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:30 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-19"}]: dispatch 2024-03-21T12:56:30.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:56:30.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-19", "mode": "writeback"}]': finished 2024-03-21T12:56:30.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: from='client.26062 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-81661-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:30.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: from='client.36059 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-81725-3","pool2":"test-rados-api-smithi043-81725-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-03-21T12:56:30.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:30.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: osdmap e189: 8 total, 8 up, 8 in 2024-03-21T12:56:30.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:30.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/580096518' entity='client.admin' cmd=[{ 2024-03-21T12:56:30.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: "prefix": "osd pool set", 2024-03-21T12:56:30.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: "pool": "PoolEIOFlag_smithi043-81661-33", 2024-03-21T12:56:30.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: "var": "eio", 2024-03-21T12:56:30.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: "val": "true" 2024-03-21T12:56:30.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: }]: dispatch 2024-03-21T12:56:30.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: from='client.26062 ' entity='client.admin' cmd=[{ 2024-03-21T12:56:30.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: "prefix": "osd pool set", 2024-03-21T12:56:30.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: "pool": "PoolEIOFlag_smithi043-81661-33", 2024-03-21T12:56:30.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: "var": "eio", 2024-03-21T12:56:30.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: "val": "true" 2024-03-21T12:56:30.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: }]: dispatch 2024-03-21T12:56:30.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: from='client.26065 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-81636-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-81636-31"}]': finished 2024-03-21T12:56:30.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:56:30.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: from='client.26062 ' entity='client.admin' cmd='[{ 2024-03-21T12:56:30.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: "prefix": "osd pool set", 2024-03-21T12:56:30.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: "pool": "PoolEIOFlag_smithi043-81661-33", 2024-03-21T12:56:30.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: "var": "eio", 2024-03-21T12:56:30.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: "val": "true" 2024-03-21T12:56:30.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: }]': finished 2024-03-21T12:56:30.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: osdmap e190: 8 total, 8 up, 8 in 2024-03-21T12:56:30.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-19"}]: dispatch 2024-03-21T12:56:30.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-19"}]: dispatch 2024-03-21T12:56:30.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:56:30.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-19", "mode": "writeback"}]': finished 2024-03-21T12:56:30.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: from='client.26062 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi043-81661-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:30.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: from='client.36059 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi043-81725-3","pool2":"test-rados-api-smithi043-81725-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-03-21T12:56:30.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:30.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: osdmap e189: 8 total, 8 up, 8 in 2024-03-21T12:56:30.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:30.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/580096518' entity='client.admin' cmd=[{ 2024-03-21T12:56:30.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: "prefix": "osd pool set", 2024-03-21T12:56:30.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: "pool": "PoolEIOFlag_smithi043-81661-33", 2024-03-21T12:56:30.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: "var": "eio", 2024-03-21T12:56:30.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: "val": "true" 2024-03-21T12:56:30.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: }]: dispatch 2024-03-21T12:56:30.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: from='client.26062 ' entity='client.admin' cmd=[{ 2024-03-21T12:56:30.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: "prefix": "osd pool set", 2024-03-21T12:56:30.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: "pool": "PoolEIOFlag_smithi043-81661-33", 2024-03-21T12:56:30.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: "var": "eio", 2024-03-21T12:56:30.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: "val": "true" 2024-03-21T12:56:30.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: }]: dispatch 2024-03-21T12:56:30.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: from='client.26065 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi043-81636-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi043-81636-31"}]': finished 2024-03-21T12:56:30.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:56:30.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: from='client.26062 ' entity='client.admin' cmd='[{ 2024-03-21T12:56:30.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: "prefix": "osd pool set", 2024-03-21T12:56:30.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: "pool": "PoolEIOFlag_smithi043-81661-33", 2024-03-21T12:56:30.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: "var": "eio", 2024-03-21T12:56:30.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: "val": "true" 2024-03-21T12:56:30.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: }]': finished 2024-03-21T12:56:30.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: osdmap e190: 8 total, 8 up, 8 in 2024-03-21T12:56:30.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-19"}]: dispatch 2024-03-21T12:56:30.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:30 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-19"}]: dispatch 2024-03-21T12:56:31.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:31 smithi155 ceph-mon[33846]: pgmap v247: 388 pgs: 32 unknown, 356 active+clean; 584 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:56:31.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:31 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:56:31.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:31 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-19"}]': finished 2024-03-21T12:56:31.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:31 smithi155 ceph-mon[33846]: osdmap e191: 8 total, 8 up, 8 in 2024-03-21T12:56:31.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:31 smithi043 ceph-mon[30062]: pgmap v247: 388 pgs: 32 unknown, 356 active+clean; 584 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:56:31.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:31 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:56:31.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:31 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-19"}]': finished 2024-03-21T12:56:31.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:31 smithi043 ceph-mon[30062]: osdmap e191: 8 total, 8 up, 8 in 2024-03-21T12:56:31.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:31 smithi043 ceph-mon[41699]: pgmap v247: 388 pgs: 32 unknown, 356 active+clean; 584 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:56:31.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:31 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:56:31.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:31 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-19"}]': finished 2024-03-21T12:56:31.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:31 smithi043 ceph-mon[41699]: osdmap e191: 8 total, 8 up, 8 in 2024-03-21T12:56:33.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:33 smithi155 ceph-mon[33846]: pgmap v250: 364 pgs: 8 unknown, 356 active+clean; 584 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:56:33.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:33 smithi155 ceph-mon[33846]: osdmap e192: 8 total, 8 up, 8 in 2024-03-21T12:56:33.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:33 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/534529931' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi043-81661-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:33.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:33 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3369657660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:33.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:33 smithi155 ceph-mon[33846]: from='client.26065 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:33.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:33 smithi155 ceph-mon[33846]: from='client.36077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi043-81661-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:33 smithi043 ceph-mon[30062]: pgmap v250: 364 pgs: 8 unknown, 356 active+clean; 584 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:56:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:33 smithi043 ceph-mon[30062]: osdmap e192: 8 total, 8 up, 8 in 2024-03-21T12:56:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:33 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/534529931' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi043-81661-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:33 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3369657660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:33 smithi043 ceph-mon[30062]: from='client.26065 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:33 smithi043 ceph-mon[30062]: from='client.36077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi043-81661-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:33.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:33 smithi043 ceph-mon[41699]: pgmap v250: 364 pgs: 8 unknown, 356 active+clean; 584 KiB data, 589 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:56:33.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:33 smithi043 ceph-mon[41699]: osdmap e192: 8 total, 8 up, 8 in 2024-03-21T12:56:33.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:33 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/534529931' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi043-81661-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:33.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:33 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3369657660' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:33.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:33 smithi043 ceph-mon[41699]: from='client.26065 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:33.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:33 smithi043 ceph-mon[41699]: from='client.36077 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi043-81661-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:34.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:34 smithi155 ceph-mon[33846]: from='client.26065 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81636-31"}]': finished 2024-03-21T12:56:34.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:34 smithi155 ceph-mon[33846]: from='client.36077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi043-81661-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:34.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:34 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3369657660' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:34.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:34 smithi155 ceph-mon[33846]: osdmap e193: 8 total, 8 up, 8 in 2024-03-21T12:56:34.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:34 smithi155 ceph-mon[33846]: from='client.26065 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:34.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:34 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:34.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:34 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:34.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:34 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:56:34.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:34 smithi155 ceph-mon[33846]: from='client.26065 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81636-31"}]': finished 2024-03-21T12:56:34.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:34 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:34.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:34 smithi155 ceph-mon[33846]: osdmap e194: 8 total, 8 up, 8 in 2024-03-21T12:56:34.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:34 smithi043 ceph-mon[30062]: from='client.26065 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81636-31"}]': finished 2024-03-21T12:56:34.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:34 smithi043 ceph-mon[30062]: from='client.36077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi043-81661-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:34.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:34 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3369657660' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:34.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:34 smithi043 ceph-mon[30062]: osdmap e193: 8 total, 8 up, 8 in 2024-03-21T12:56:34.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:34 smithi043 ceph-mon[30062]: from='client.26065 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:34.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:34 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:34.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:34 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:34.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:34 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:56:34.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:34 smithi043 ceph-mon[30062]: from='client.26065 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81636-31"}]': finished 2024-03-21T12:56:34.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:34 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:34.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:34 smithi043 ceph-mon[30062]: osdmap e194: 8 total, 8 up, 8 in 2024-03-21T12:56:34.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:34 smithi043 ceph-mon[41699]: from='client.26065 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi043-81636-31"}]': finished 2024-03-21T12:56:34.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:34 smithi043 ceph-mon[41699]: from='client.36077 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi043-81661-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:34.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:34 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3369657660' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:34.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:34 smithi043 ceph-mon[41699]: osdmap e193: 8 total, 8 up, 8 in 2024-03-21T12:56:34.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:34 smithi043 ceph-mon[41699]: from='client.26065 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81636-31"}]: dispatch 2024-03-21T12:56:34.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:34 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:34.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:34 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:34.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:34 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:56:34.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:34 smithi043 ceph-mon[41699]: from='client.26065 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi043-81636-31"}]': finished 2024-03-21T12:56:34.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:34 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:34.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:34 smithi043 ceph-mon[41699]: osdmap e194: 8 total, 8 up, 8 in 2024-03-21T12:56:35.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:35 smithi155 ceph-mon[33846]: pgmap v253: 388 pgs: 32 unknown, 32 creating+peering, 324 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:56:35.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:35 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:35.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:35 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:35.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:35 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1740120350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:35.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:35 smithi155 ceph-mon[33846]: from='client.26083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:35.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:35 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1740120350' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:35.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:35 smithi155 ceph-mon[33846]: from='client.26083 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:35.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:35 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1740120350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-81636-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:35.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:35 smithi155 ceph-mon[33846]: from='client.26083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-81636-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:35.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:35 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-21", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:56:35.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:35 smithi155 ceph-mon[33846]: from='client.26083 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-81636-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:35.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:35 smithi155 ceph-mon[33846]: osdmap e195: 8 total, 8 up, 8 in 2024-03-21T12:56:35.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:35 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-21"}]: dispatch 2024-03-21T12:56:35.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:35 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1740120350' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-81636-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:35.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:35 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2838655352' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-81661-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:35.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:35 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-21"}]: dispatch 2024-03-21T12:56:35.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:35 smithi155 ceph-mon[33846]: from='client.26083 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-81636-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:35.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:35 smithi155 ceph-mon[33846]: from='client.26080 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-81661-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[30062]: pgmap v253: 388 pgs: 32 unknown, 32 creating+peering, 324 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:56:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1740120350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[30062]: from='client.26083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1740120350' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[30062]: from='client.26083 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1740120350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-81636-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[30062]: from='client.26083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-81636-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:35.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-21", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:56:35.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[30062]: from='client.26083 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-81636-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:35.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[30062]: osdmap e195: 8 total, 8 up, 8 in 2024-03-21T12:56:35.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-21"}]: dispatch 2024-03-21T12:56:35.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1740120350' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-81636-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:35.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2838655352' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-81661-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:35.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-21"}]: dispatch 2024-03-21T12:56:35.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[30062]: from='client.26083 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-81636-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:35.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[30062]: from='client.26080 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-81661-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:35.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[41699]: pgmap v253: 388 pgs: 32 unknown, 32 creating+peering, 324 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:56:35.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:35.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:35.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1740120350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:35.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[41699]: from='client.26083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:35.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1740120350' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:35.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[41699]: from='client.26083 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:35.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1740120350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-81636-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:35.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[41699]: from='client.26083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-81636-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:35.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-21", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:56:35.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[41699]: from='client.26083 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi043-81636-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:35.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[41699]: osdmap e195: 8 total, 8 up, 8 in 2024-03-21T12:56:35.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-21"}]: dispatch 2024-03-21T12:56:35.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1740120350' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-81636-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:35.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2838655352' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-81661-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:35.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-21"}]: dispatch 2024-03-21T12:56:35.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[41699]: from='client.26083 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-81636-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:35.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:35 smithi043 ceph-mon[41699]: from='client.26080 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-81661-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:37.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:37 smithi155 ceph-mon[33846]: pgmap v256: 388 pgs: 64 unknown, 324 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:56:37.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:37 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-21"}]': finished 2024-03-21T12:56:37.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:37 smithi155 ceph-mon[33846]: from='client.26080 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-81661-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:37.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:37 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-21", "mode": "writeback"}]: dispatch 2024-03-21T12:56:37.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:37 smithi155 ceph-mon[33846]: osdmap e196: 8 total, 8 up, 8 in 2024-03-21T12:56:37.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:37 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-21", "mode": "writeback"}]: dispatch 2024-03-21T12:56:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:37 smithi043 ceph-mon[30062]: pgmap v256: 388 pgs: 64 unknown, 324 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:56:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:37 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-21"}]': finished 2024-03-21T12:56:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:37 smithi043 ceph-mon[30062]: from='client.26080 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-81661-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:37 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-21", "mode": "writeback"}]: dispatch 2024-03-21T12:56:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:37 smithi043 ceph-mon[30062]: osdmap e196: 8 total, 8 up, 8 in 2024-03-21T12:56:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:37 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-21", "mode": "writeback"}]: dispatch 2024-03-21T12:56:37.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:37 smithi043 ceph-mon[41699]: pgmap v256: 388 pgs: 64 unknown, 324 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:56:37.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:37 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-21"}]': finished 2024-03-21T12:56:37.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:37 smithi043 ceph-mon[41699]: from='client.26080 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi043-81661-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:37.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:37 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-21", "mode": "writeback"}]: dispatch 2024-03-21T12:56:37.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:37 smithi043 ceph-mon[41699]: osdmap e196: 8 total, 8 up, 8 in 2024-03-21T12:56:37.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:37 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-21", "mode": "writeback"}]: dispatch 2024-03-21T12:56:38.399 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:56:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:56:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:56:38.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:56:38 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:56:38.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:38 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:56:38.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:38 smithi155 ceph-mon[33846]: from='client.26083 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-81636-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-81636-32"}]': finished 2024-03-21T12:56:38.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:38 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-21", "mode": "writeback"}]': finished 2024-03-21T12:56:38.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:38 smithi155 ceph-mon[33846]: osdmap e197: 8 total, 8 up, 8 in 2024-03-21T12:56:38.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:38 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:38.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:38 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:38.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:38 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:56:38.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:38 smithi043 ceph-mon[30062]: from='client.26083 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-81636-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-81636-32"}]': finished 2024-03-21T12:56:38.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:38 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-21", "mode": "writeback"}]': finished 2024-03-21T12:56:38.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:38 smithi043 ceph-mon[30062]: osdmap e197: 8 total, 8 up, 8 in 2024-03-21T12:56:38.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:38 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:38.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:38 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:38.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:38 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:56:38.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:38 smithi043 ceph-mon[41699]: from='client.26083 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi043-81636-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi043-81636-32"}]': finished 2024-03-21T12:56:38.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:38 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-21", "mode": "writeback"}]': finished 2024-03-21T12:56:38.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:38 smithi043 ceph-mon[41699]: osdmap e197: 8 total, 8 up, 8 in 2024-03-21T12:56:38.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:38 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:38.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:38 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:39.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:39 smithi155 ceph-mon[33846]: pgmap v259: 364 pgs: 40 unknown, 324 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:56:39.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:39 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:56:39.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:39 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:56:39.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:39 smithi155 ceph-mon[33846]: osdmap e198: 8 total, 8 up, 8 in 2024-03-21T12:56:39.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:39 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-21"}]: dispatch 2024-03-21T12:56:39.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:39 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-21"}]: dispatch 2024-03-21T12:56:39.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:39 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3184549328' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81661-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:39.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:39 smithi043 ceph-mon[30062]: pgmap v259: 364 pgs: 40 unknown, 324 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:56:39.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:39 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:56:39.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:39 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:56:39.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:39 smithi043 ceph-mon[30062]: osdmap e198: 8 total, 8 up, 8 in 2024-03-21T12:56:39.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:39 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-21"}]: dispatch 2024-03-21T12:56:39.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:39 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-21"}]: dispatch 2024-03-21T12:56:39.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:39 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3184549328' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81661-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:39.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:39 smithi043 ceph-mon[41699]: pgmap v259: 364 pgs: 40 unknown, 324 active+clean; 584 KiB data, 570 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:56:39.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:39 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:56:39.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:39 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:56:39.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:39 smithi043 ceph-mon[41699]: osdmap e198: 8 total, 8 up, 8 in 2024-03-21T12:56:39.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:39 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-21"}]: dispatch 2024-03-21T12:56:39.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:39 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-21"}]: dispatch 2024-03-21T12:56:39.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:39 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3184549328' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81661-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:40.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:40 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:56:40.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:40 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-21"}]': finished 2024-03-21T12:56:40.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:40 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3184549328' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81661-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:40.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:40 smithi155 ceph-mon[33846]: osdmap e199: 8 total, 8 up, 8 in 2024-03-21T12:56:40.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:40 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1740120350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:40.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:40 smithi155 ceph-mon[33846]: from='client.26083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:40.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:40 smithi155 ceph-mon[33846]: from='client.26083 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81636-32"}]': finished 2024-03-21T12:56:40.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:40 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:56:40.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:40 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-21"}]': finished 2024-03-21T12:56:40.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:40 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3184549328' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81661-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:40.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:40 smithi043 ceph-mon[30062]: osdmap e199: 8 total, 8 up, 8 in 2024-03-21T12:56:40.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:40 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1740120350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:40.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:40 smithi043 ceph-mon[30062]: from='client.26083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:40.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:40 smithi043 ceph-mon[30062]: from='client.26083 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81636-32"}]': finished 2024-03-21T12:56:40.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:40 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:56:40.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:40 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-21"}]': finished 2024-03-21T12:56:40.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:40 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3184549328' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81661-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:40.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:40 smithi043 ceph-mon[41699]: osdmap e199: 8 total, 8 up, 8 in 2024-03-21T12:56:40.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:40 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1740120350' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:40.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:40 smithi043 ceph-mon[41699]: from='client.26083 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:40.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:40 smithi043 ceph-mon[41699]: from='client.26083 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi043-81636-32"}]': finished 2024-03-21T12:56:41.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[30062]: pgmap v262: 388 pgs: 32 unknown, 356 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-21T12:56:41.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[30062]: osdmap e200: 8 total, 8 up, 8 in 2024-03-21T12:56:41.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1740120350' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:41.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:41.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[30062]: from='client.26083 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:41.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4142224133' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81661-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:41.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[30062]: from='client.26083 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81636-32"}]': finished 2024-03-21T12:56:41.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4142224133' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81661-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:41.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[30062]: osdmap e201: 8 total, 8 up, 8 in 2024-03-21T12:56:41.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:41.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:41.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3511558128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:41.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[30062]: from='client.26101 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:41.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3511558128' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:41.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[30062]: from='client.26101 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:41.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3511558128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-81636-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:41.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[30062]: from='client.26101 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-81636-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:41.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[41699]: pgmap v262: 388 pgs: 32 unknown, 356 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-21T12:56:41.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[41699]: osdmap e200: 8 total, 8 up, 8 in 2024-03-21T12:56:41.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1740120350' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:41.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:41.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[41699]: from='client.26083 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:41.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4142224133' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81661-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:41.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[41699]: from='client.26083 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81636-32"}]': finished 2024-03-21T12:56:41.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4142224133' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81661-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:41.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[41699]: osdmap e201: 8 total, 8 up, 8 in 2024-03-21T12:56:41.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:41.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:41.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3511558128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:41.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[41699]: from='client.26101 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:41.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3511558128' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:41.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[41699]: from='client.26101 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:41.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3511558128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-81636-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:41.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:41 smithi043 ceph-mon[41699]: from='client.26101 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-81636-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:41 smithi155 ceph-mon[33846]: pgmap v262: 388 pgs: 32 unknown, 356 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-21T12:56:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:41 smithi155 ceph-mon[33846]: osdmap e200: 8 total, 8 up, 8 in 2024-03-21T12:56:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1740120350' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:41 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:42.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:41 smithi155 ceph-mon[33846]: from='client.26083 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81636-32"}]: dispatch 2024-03-21T12:56:42.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4142224133' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81661-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:42.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:41 smithi155 ceph-mon[33846]: from='client.26083 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi043-81636-32"}]': finished 2024-03-21T12:56:42.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4142224133' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81661-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:42.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:41 smithi155 ceph-mon[33846]: osdmap e201: 8 total, 8 up, 8 in 2024-03-21T12:56:42.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:42.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:41 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:42.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3511558128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:42.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:41 smithi155 ceph-mon[33846]: from='client.26101 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:42.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3511558128' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:42.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:41 smithi155 ceph-mon[33846]: from='client.26101 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:42.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3511558128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-81636-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:42.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:41 smithi155 ceph-mon[33846]: from='client.26101 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-81636-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:43.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:43 smithi155 ceph-mon[33846]: pgmap v265: 420 pgs: 96 unknown, 324 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T12:56:43.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:43 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:43.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:43 smithi155 ceph-mon[33846]: from='client.26101 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-81636-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:43.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:43 smithi155 ceph-mon[33846]: osdmap e202: 8 total, 8 up, 8 in 2024-03-21T12:56:43.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3511558128' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-81636-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:43.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1000451871' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81661-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:43.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:43 smithi155 ceph-mon[33846]: from='client.26101 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-81636-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:43.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:43 smithi043 ceph-mon[30062]: pgmap v265: 420 pgs: 96 unknown, 324 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T12:56:43.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:43 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:43.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:43 smithi043 ceph-mon[30062]: from='client.26101 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-81636-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:43.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:43 smithi043 ceph-mon[30062]: osdmap e202: 8 total, 8 up, 8 in 2024-03-21T12:56:43.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3511558128' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-81636-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:43.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1000451871' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81661-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:43.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:43 smithi043 ceph-mon[30062]: from='client.26101 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-81636-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:43.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:43 smithi043 ceph-mon[41699]: pgmap v265: 420 pgs: 96 unknown, 324 active+clean; 584 KiB data, 562 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T12:56:43.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:43 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:43.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:43 smithi043 ceph-mon[41699]: from='client.26101 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi043-81636-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:43.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:43 smithi043 ceph-mon[41699]: osdmap e202: 8 total, 8 up, 8 in 2024-03-21T12:56:43.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3511558128' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-81636-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:43.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1000451871' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81661-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:43.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:43 smithi043 ceph-mon[41699]: from='client.26101 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-81636-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:44.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:44 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1000451871' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81661-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:44.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:44 smithi155 ceph-mon[33846]: osdmap e203: 8 total, 8 up, 8 in 2024-03-21T12:56:44.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:44 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:44.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:44 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:44.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:44 smithi155 ceph-mon[33846]: from='client.26101 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-81636-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-81636-33"}]': finished 2024-03-21T12:56:44.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:44 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-23", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:56:44.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:44 smithi155 ceph-mon[33846]: osdmap e204: 8 total, 8 up, 8 in 2024-03-21T12:56:44.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:44 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-23"}]: dispatch 2024-03-21T12:56:44.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:44 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-23"}]: dispatch 2024-03-21T12:56:44.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:44 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1000451871' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81661-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:44.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:44 smithi043 ceph-mon[41699]: osdmap e203: 8 total, 8 up, 8 in 2024-03-21T12:56:44.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:44 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:44.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:44 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:44.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:44 smithi043 ceph-mon[41699]: from='client.26101 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-81636-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-81636-33"}]': finished 2024-03-21T12:56:44.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:44 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-23", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:56:44.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:44 smithi043 ceph-mon[41699]: osdmap e204: 8 total, 8 up, 8 in 2024-03-21T12:56:44.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:44 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-23"}]: dispatch 2024-03-21T12:56:44.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:44 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-23"}]: dispatch 2024-03-21T12:56:44.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:44 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1000451871' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi043-81661-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:44.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:44 smithi043 ceph-mon[30062]: osdmap e203: 8 total, 8 up, 8 in 2024-03-21T12:56:44.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:44 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:44.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:44 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:44.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:44 smithi043 ceph-mon[30062]: from='client.26101 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi043-81636-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi043-81636-33"}]': finished 2024-03-21T12:56:44.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:44 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-23", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:56:44.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:44 smithi043 ceph-mon[30062]: osdmap e204: 8 total, 8 up, 8 in 2024-03-21T12:56:44.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:44 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-23"}]: dispatch 2024-03-21T12:56:44.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:44 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-23"}]: dispatch 2024-03-21T12:56:45.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:45 smithi155 ceph-mon[33846]: pgmap v268: 452 pgs: 32 creating+peering, 420 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 0 op/s 2024-03-21T12:56:45.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:45 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-23"}]': finished 2024-03-21T12:56:45.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:45 smithi155 ceph-mon[33846]: osdmap e205: 8 total, 8 up, 8 in 2024-03-21T12:56:45.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:45 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-23", "mode": "writeback"}]: dispatch 2024-03-21T12:56:45.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:45 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-23", "mode": "writeback"}]: dispatch 2024-03-21T12:56:45.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:45 smithi043 ceph-mon[30062]: pgmap v268: 452 pgs: 32 creating+peering, 420 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 0 op/s 2024-03-21T12:56:45.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:45 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-23"}]': finished 2024-03-21T12:56:45.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:45 smithi043 ceph-mon[30062]: osdmap e205: 8 total, 8 up, 8 in 2024-03-21T12:56:45.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-23", "mode": "writeback"}]: dispatch 2024-03-21T12:56:45.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:45 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-23", "mode": "writeback"}]: dispatch 2024-03-21T12:56:45.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:45 smithi043 ceph-mon[41699]: pgmap v268: 452 pgs: 32 creating+peering, 420 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 0 op/s 2024-03-21T12:56:45.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:45 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-23"}]': finished 2024-03-21T12:56:45.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:45 smithi043 ceph-mon[41699]: osdmap e205: 8 total, 8 up, 8 in 2024-03-21T12:56:45.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:45 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-23", "mode": "writeback"}]: dispatch 2024-03-21T12:56:45.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:45 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-23", "mode": "writeback"}]: dispatch 2024-03-21T12:56:46.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:46 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:56:46.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:46 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-23", "mode": "writeback"}]': finished 2024-03-21T12:56:46.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:46 smithi043 ceph-mon[30062]: osdmap e206: 8 total, 8 up, 8 in 2024-03-21T12:56:46.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3511558128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:46.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:46 smithi043 ceph-mon[30062]: from='client.26101 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:46.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:46 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:56:46.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:46 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-23", "mode": "writeback"}]': finished 2024-03-21T12:56:46.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:46 smithi043 ceph-mon[41699]: osdmap e206: 8 total, 8 up, 8 in 2024-03-21T12:56:46.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3511558128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:46.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:46 smithi043 ceph-mon[41699]: from='client.26101 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:47.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:46 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:56:47.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:46 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-23", "mode": "writeback"}]': finished 2024-03-21T12:56:47.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:46 smithi155 ceph-mon[33846]: osdmap e206: 8 total, 8 up, 8 in 2024-03-21T12:56:47.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3511558128' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:47.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:46 smithi155 ceph-mon[33846]: from='client.26101 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:47.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:47 smithi043 ceph-mon[30062]: pgmap v271: 396 pgs: 8 unknown, 388 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T12:56:47.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:47 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:47.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:47 smithi043 ceph-mon[30062]: from='client.26101 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-81636-33"}]': finished 2024-03-21T12:56:47.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:47 smithi043 ceph-mon[30062]: osdmap e207: 8 total, 8 up, 8 in 2024-03-21T12:56:47.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3511558128' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:47.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:47 smithi043 ceph-mon[30062]: from='client.26101 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:47.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1425806600' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:47.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:47 smithi043 ceph-mon[30062]: from='client.36116 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:47.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:47.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:47 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:47.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:47 smithi043 ceph-mon[41699]: pgmap v271: 396 pgs: 8 unknown, 388 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T12:56:47.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:47 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:47.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:47 smithi043 ceph-mon[41699]: from='client.26101 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-81636-33"}]': finished 2024-03-21T12:56:47.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:47 smithi043 ceph-mon[41699]: osdmap e207: 8 total, 8 up, 8 in 2024-03-21T12:56:47.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3511558128' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:47.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:47 smithi043 ceph-mon[41699]: from='client.26101 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:47.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1425806600' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:47.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:47 smithi043 ceph-mon[41699]: from='client.36116 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:47.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:47.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:47 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:47 smithi155 ceph-mon[33846]: pgmap v271: 396 pgs: 8 unknown, 388 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T12:56:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:47 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:47 smithi155 ceph-mon[33846]: from='client.26101 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi043-81636-33"}]': finished 2024-03-21T12:56:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:47 smithi155 ceph-mon[33846]: osdmap e207: 8 total, 8 up, 8 in 2024-03-21T12:56:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3511558128' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:47 smithi155 ceph-mon[33846]: from='client.26101 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-81636-33"}]: dispatch 2024-03-21T12:56:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1425806600' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:47 smithi155 ceph-mon[33846]: from='client.36116 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:48.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:48.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:47 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:48.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:56:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:56:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:56:48.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:56:48 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:56:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:49 smithi155 ceph-mon[33846]: pgmap v274: 388 pgs: 32 unknown, 356 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:56:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:49 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:56:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:49 smithi155 ceph-mon[33846]: from='client.26101 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-81636-33"}]': finished 2024-03-21T12:56:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:49 smithi155 ceph-mon[33846]: from='client.36116 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:49 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:56:49.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:49 smithi155 ceph-mon[33846]: osdmap e208: 8 total, 8 up, 8 in 2024-03-21T12:56:49.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:49 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-23"}]: dispatch 2024-03-21T12:56:49.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:49 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-23"}]: dispatch 2024-03-21T12:56:49.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:49 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4002592202' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:49.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:49 smithi155 ceph-mon[33846]: from='client.36122 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:49.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:49 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4002592202' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:49.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:49 smithi155 ceph-mon[33846]: from='client.36122 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:49.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:49 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4002592202' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-81636-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:49.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:49 smithi155 ceph-mon[33846]: from='client.36122 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-81636-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:49.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:49 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:56:49.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:49 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:56:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[30062]: pgmap v274: 388 pgs: 32 unknown, 356 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:56:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:56:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[30062]: from='client.26101 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-81636-33"}]': finished 2024-03-21T12:56:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[30062]: from='client.36116 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:56:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[30062]: osdmap e208: 8 total, 8 up, 8 in 2024-03-21T12:56:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-23"}]: dispatch 2024-03-21T12:56:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-23"}]: dispatch 2024-03-21T12:56:49.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4002592202' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:49.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[30062]: from='client.36122 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:49.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4002592202' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:49.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[30062]: from='client.36122 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:49.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4002592202' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-81636-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:49.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[30062]: from='client.36122 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-81636-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:49.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:56:49.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:56:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[41699]: pgmap v274: 388 pgs: 32 unknown, 356 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:56:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:56:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[41699]: from='client.26101 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi043-81636-33"}]': finished 2024-03-21T12:56:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[41699]: from='client.36116 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:49.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:56:49.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[41699]: osdmap e208: 8 total, 8 up, 8 in 2024-03-21T12:56:49.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-23"}]: dispatch 2024-03-21T12:56:49.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-23"}]: dispatch 2024-03-21T12:56:49.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4002592202' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:49.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[41699]: from='client.36122 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:49.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4002592202' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:49.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[41699]: from='client.36122 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:49.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4002592202' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-81636-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:49.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[41699]: from='client.36122 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-81636-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:49.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:56:49.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:49 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:56:50.550 INFO:tasks.workunit.client.0.smithi043.stdout: 0x558552520480 2024-03-21T12:56:50.550 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525205b0 2024-03-21T12:56:50.550 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525206c0 2024-03-21T12:56:50.550 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525207f0 2024-03-21T12:56:50.550 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552520950 2024-03-21T12:56:50.551 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251ffd0 2024-03-21T12:56:50.551 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552520100 2024-03-21T12:56:50.551 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552520250 2024-03-21T12:56:50.551 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552520a80 2024-03-21T12:56:50.551 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552520f50 2024-03-21T12:56:50.551 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552521080 2024-03-21T12:56:50.551 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552520360 2024-03-21T12:56:50.551 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552520bb0 2024-03-21T12:56:50.551 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552520d00 2024-03-21T12:56:50.551 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552520e30 2024-03-21T12:56:50.551 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552521660 2024-03-21T12:56:50.551 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525211b0 2024-03-21T12:56:50.551 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251e130 2024-03-21T12:56:50.551 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855251e260 2024-03-21T12:56:50.551 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552521790 2024-03-21T12:56:50.551 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552521c80 2024-03-21T12:56:50.552 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552521d90 2024-03-21T12:56:50.552 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525212c0 2024-03-21T12:56:50.552 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552521430 2024-03-21T12:56:50.552 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552522350 2024-03-21T12:56:50.552 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552521ea0 2024-03-21T12:56:50.552 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552521ff0 2024-03-21T12:56:50.552 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552522120 2024-03-21T12:56:50.552 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552522460 2024-03-21T12:56:50.552 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552522910 2024-03-21T12:56:50.552 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552522a60 2024-03-21T12:56:50.552 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552523020 2024-03-21T12:56:50.552 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552522570 2024-03-21T12:56:50.552 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525226a0 2024-03-21T12:56:50.552 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525227d0 2024-03-21T12:56:50.552 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552522230 2024-03-21T12:56:50.552 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552522b70 2024-03-21T12:56:50.552 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525218e0 2024-03-21T12:56:50.553 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552521a30 2024-03-21T12:56:50.553 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552521b40 2024-03-21T12:56:50.553 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525235e0 2024-03-21T12:56:50.553 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552523130 2024-03-21T12:56:50.553 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552523730 2024-03-21T12:56:50.553 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552523840 2024-03-21T12:56:50.553 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552523e00 2024-03-21T12:56:50.553 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552523240 2024-03-21T12:56:50.553 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552523350 2024-03-21T12:56:50.553 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525234a0 2024-03-21T12:56:50.553 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552523f50 2024-03-21T12:56:50.553 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552523950 2024-03-21T12:56:50.553 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552522c80 2024-03-21T12:56:50.553 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552522dd0 2024-03-21T12:56:50.553 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525240a0 2024-03-21T12:56:50.553 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552524590 2024-03-21T12:56:50.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525246c0 2024-03-21T12:56:50.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552522f00 2024-03-21T12:56:50.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525247f0 2024-03-21T12:56:50.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552524980 2024-03-21T12:56:50.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552524f80 2024-03-21T12:56:50.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552523a60 2024-03-21T12:56:50.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552523b90 2024-03-21T12:56:50.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552523ce0 2024-03-21T12:56:50.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525250d0 2024-03-21T12:56:50.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552524ad0 2024-03-21T12:56:50.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552524c00 2024-03-21T12:56:50.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552524d50 2024-03-21T12:56:50.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552525240 2024-03-21T12:56:50.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552525730 2024-03-21T12:56:50.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552525860 2024-03-21T12:56:50.554 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552525e40 2024-03-21T12:56:50.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552525390 2024-03-21T12:56:50.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525254a0 2024-03-21T12:56:50.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525255d0 2024-03-21T12:56:50.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552524e60 2024-03-21T12:56:50.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552525990 2024-03-21T12:56:50.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552525b00 2024-03-21T12:56:50.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552525c30 2024-03-21T12:56:50.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552525f90 2024-03-21T12:56:50.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552526460 2024-03-21T12:56:50.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552526590 2024-03-21T12:56:50.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552526b70 2024-03-21T12:56:50.555 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525260c0 2024-03-21T12:56:51.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:51 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:56:51.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:51 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-23"}]': finished 2024-03-21T12:56:51.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:51 smithi155 ceph-mon[33846]: from='client.36122 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-81636-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:51.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:51 smithi155 ceph-mon[33846]: osdmap e209: 8 total, 8 up, 8 in 2024-03-21T12:56:51.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:51 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4002592202' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-81636-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:51.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:51 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2900109253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:51.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:51 smithi155 ceph-mon[33846]: from='client.36122 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-81636-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:51.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:51 smithi155 ceph-mon[33846]: from='client.36128 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:51.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:51 smithi155 ceph-mon[33846]: from='client.36128 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:51.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:51 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:56:51.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:51 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-23"}]': finished 2024-03-21T12:56:51.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:51 smithi043 ceph-mon[30062]: from='client.36122 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-81636-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:51.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:51 smithi043 ceph-mon[30062]: osdmap e209: 8 total, 8 up, 8 in 2024-03-21T12:56:51.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:51 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4002592202' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-81636-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:51.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:51 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2900109253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:51.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:51 smithi043 ceph-mon[30062]: from='client.36122 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-81636-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:51.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:51 smithi043 ceph-mon[30062]: from='client.36128 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:51.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:51 smithi043 ceph-mon[30062]: from='client.36128 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:51.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:51 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:56:51.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:51 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-23"}]': finished 2024-03-21T12:56:51.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:51 smithi043 ceph-mon[41699]: from='client.36122 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi043-81636-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:51.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:51 smithi043 ceph-mon[41699]: osdmap e209: 8 total, 8 up, 8 in 2024-03-21T12:56:51.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:51 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4002592202' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-81636-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:51.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:51 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2900109253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:51.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:51 smithi043 ceph-mon[41699]: from='client.36122 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-81636-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:51.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:51 smithi043 ceph-mon[41699]: from='client.36128 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:51.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:51 smithi043 ceph-mon[41699]: from='client.36128 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:52.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:52 smithi155 ceph-mon[33846]: pgmap v277: 420 pgs: 32 unknown, 32 creating+peering, 356 active+clean; 584 KiB data, 548 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-21T12:56:52.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:52 smithi155 ceph-mon[33846]: osdmap e210: 8 total, 8 up, 8 in 2024-03-21T12:56:52.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:52 smithi155 ceph-mon[33846]: from='client.36122 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-81636-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-81636-34"}]': finished 2024-03-21T12:56:52.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:52 smithi155 ceph-mon[33846]: osdmap e211: 8 total, 8 up, 8 in 2024-03-21T12:56:52.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:52.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:52 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:52.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4264238526' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:52.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:52 smithi155 ceph-mon[33846]: from='client.36134 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:52.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:52 smithi043 ceph-mon[30062]: pgmap v277: 420 pgs: 32 unknown, 32 creating+peering, 356 active+clean; 584 KiB data, 548 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-21T12:56:52.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:52 smithi043 ceph-mon[30062]: osdmap e210: 8 total, 8 up, 8 in 2024-03-21T12:56:52.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:52 smithi043 ceph-mon[30062]: from='client.36122 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-81636-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-81636-34"}]': finished 2024-03-21T12:56:52.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:52 smithi043 ceph-mon[30062]: osdmap e211: 8 total, 8 up, 8 in 2024-03-21T12:56:52.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:52.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:52 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:52.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4264238526' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:52.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:52 smithi043 ceph-mon[30062]: from='client.36134 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:52.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:52 smithi043 ceph-mon[41699]: pgmap v277: 420 pgs: 32 unknown, 32 creating+peering, 356 active+clean; 584 KiB data, 548 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-03-21T12:56:52.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:52 smithi043 ceph-mon[41699]: osdmap e210: 8 total, 8 up, 8 in 2024-03-21T12:56:52.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:52 smithi043 ceph-mon[41699]: from='client.36122 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi043-81636-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi043-81636-34"}]': finished 2024-03-21T12:56:52.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:52 smithi043 ceph-mon[41699]: osdmap e211: 8 total, 8 up, 8 in 2024-03-21T12:56:52.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:52.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:52 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:52.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4264238526' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:52.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:52 smithi043 ceph-mon[41699]: from='client.36134 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:53.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:53 smithi155 ceph-mon[33846]: pgmap v280: 460 pgs: 104 unknown, 32 creating+peering, 324 active+clean; 584 KiB data, 548 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T12:56:53.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:53 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:53.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:53 smithi155 ceph-mon[33846]: from='client.36134 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:53.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:53 smithi155 ceph-mon[33846]: osdmap e212: 8 total, 8 up, 8 in 2024-03-21T12:56:53.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:53 smithi155 ceph-mon[33846]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:53.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:53 smithi043 ceph-mon[30062]: pgmap v280: 460 pgs: 104 unknown, 32 creating+peering, 324 active+clean; 584 KiB data, 548 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T12:56:53.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:53 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:53.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:53 smithi043 ceph-mon[30062]: from='client.36134 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:53.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:53 smithi043 ceph-mon[30062]: osdmap e212: 8 total, 8 up, 8 in 2024-03-21T12:56:53.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:53 smithi043 ceph-mon[30062]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:53.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:53 smithi043 ceph-mon[41699]: pgmap v280: 460 pgs: 104 unknown, 32 creating+peering, 324 active+clean; 584 KiB data, 548 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T12:56:53.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:53 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:53.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:53 smithi043 ceph-mon[41699]: from='client.36134 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:53.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:53 smithi043 ceph-mon[41699]: osdmap e212: 8 total, 8 up, 8 in 2024-03-21T12:56:53.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:53 smithi043 ceph-mon[41699]: Health check update: 8 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:54.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:54 smithi155 ceph-mon[33846]: osdmap e213: 8 total, 8 up, 8 in 2024-03-21T12:56:54.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:54 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4002592202' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:54.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:54 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3602334124' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:54.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:54 smithi155 ceph-mon[33846]: from='client.36122 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:54.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:54 smithi155 ceph-mon[33846]: from='client.36140 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:54.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:54 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:54.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:54 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:54.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:54 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:56:54.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:54 smithi155 ceph-mon[33846]: from='client.36122 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81636-34"}]': finished 2024-03-21T12:56:54.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:54 smithi155 ceph-mon[33846]: from='client.36140 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:54.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:54 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-25", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:56:54.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:54 smithi155 ceph-mon[33846]: osdmap e214: 8 total, 8 up, 8 in 2024-03-21T12:56:54.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:54 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4002592202' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:54.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:54 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-25"}]: dispatch 2024-03-21T12:56:54.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:54 smithi155 ceph-mon[33846]: from='client.36122 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:54.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[30062]: osdmap e213: 8 total, 8 up, 8 in 2024-03-21T12:56:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4002592202' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3602334124' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[30062]: from='client.36122 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[30062]: from='client.36140 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:56:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[30062]: from='client.36122 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81636-34"}]': finished 2024-03-21T12:56:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[30062]: from='client.36140 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-25", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:56:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[30062]: osdmap e214: 8 total, 8 up, 8 in 2024-03-21T12:56:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4002592202' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-25"}]: dispatch 2024-03-21T12:56:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[30062]: from='client.36122 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:54.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[41699]: osdmap e213: 8 total, 8 up, 8 in 2024-03-21T12:56:54.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4002592202' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:54.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3602334124' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:54.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[41699]: from='client.36122 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:54.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[41699]: from='client.36140 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:54.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:54.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:56:54.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:56:54.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[41699]: from='client.36122 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi043-81636-34"}]': finished 2024-03-21T12:56:54.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[41699]: from='client.36140 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:54.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-25", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:56:54.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[41699]: osdmap e214: 8 total, 8 up, 8 in 2024-03-21T12:56:54.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4002592202' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:54.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-25"}]: dispatch 2024-03-21T12:56:54.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:54 smithi043 ceph-mon[41699]: from='client.36122 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81636-34"}]: dispatch 2024-03-21T12:56:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:55 smithi155 ceph-mon[33846]: pgmap v283: 484 pgs: 32 unknown, 452 active+clean; 584 KiB data, 548 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:56:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:55 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-25"}]: dispatch 2024-03-21T12:56:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:55 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:56:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:55 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:56:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:55 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:56:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:55 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:56:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:55 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:56:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:55 smithi155 ceph-mon[33846]: from='client.36122 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81636-34"}]': finished 2024-03-21T12:56:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:55 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-25"}]': finished 2024-03-21T12:56:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:55 smithi155 ceph-mon[33846]: osdmap e215: 8 total, 8 up, 8 in 2024-03-21T12:56:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:55 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-25", "mode": "writeback"}]: dispatch 2024-03-21T12:56:55.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:55 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-25", "mode": "writeback"}]: dispatch 2024-03-21T12:56:55.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:55 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/252865393' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:55.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:55 smithi155 ceph-mon[33846]: from='client.26140 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:55.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:55 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/739869095' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:56:55.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:55 smithi155 ceph-mon[33846]: from='client.36152 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:56:55.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:55 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/739869095' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:56:55.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:55 smithi155 ceph-mon[33846]: from='client.36152 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:56:55.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:55 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/739869095' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-81636-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:55.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:55 smithi155 ceph-mon[33846]: from='client.36152 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-81636-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[30062]: pgmap v283: 484 pgs: 32 unknown, 452 active+clean; 584 KiB data, 548 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:56:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-25"}]: dispatch 2024-03-21T12:56:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:56:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:56:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:56:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:56:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:56:55.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[30062]: from='client.36122 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81636-34"}]': finished 2024-03-21T12:56:55.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-25"}]': finished 2024-03-21T12:56:55.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[30062]: osdmap e215: 8 total, 8 up, 8 in 2024-03-21T12:56:55.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-25", "mode": "writeback"}]: dispatch 2024-03-21T12:56:55.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-25", "mode": "writeback"}]: dispatch 2024-03-21T12:56:55.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/252865393' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:55.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[30062]: from='client.26140 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:55.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/739869095' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:56:55.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[30062]: from='client.36152 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:56:55.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/739869095' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:56:55.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[30062]: from='client.36152 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:56:55.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/739869095' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-81636-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:55.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[30062]: from='client.36152 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-81636-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:55.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[41699]: pgmap v283: 484 pgs: 32 unknown, 452 active+clean; 584 KiB data, 548 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:56:55.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-25"}]: dispatch 2024-03-21T12:56:55.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:56:55.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:56:55.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:56:55.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:56:55.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:56:55.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[41699]: from='client.36122 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi043-81636-34"}]': finished 2024-03-21T12:56:55.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-25"}]': finished 2024-03-21T12:56:55.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[41699]: osdmap e215: 8 total, 8 up, 8 in 2024-03-21T12:56:55.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-25", "mode": "writeback"}]: dispatch 2024-03-21T12:56:55.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-25", "mode": "writeback"}]: dispatch 2024-03-21T12:56:55.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/252865393' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:55.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[41699]: from='client.26140 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:56:55.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/739869095' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:56:55.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[41699]: from='client.36152 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:56:55.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/739869095' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:56:55.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[41699]: from='client.36152 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:56:55.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/739869095' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-81636-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:55.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:55 smithi043 ceph-mon[41699]: from='client.36152 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-81636-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:56:56.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:56 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:56:56.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:56 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-25", "mode": "writeback"}]': finished 2024-03-21T12:56:56.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:56 smithi155 ceph-mon[33846]: from='client.26140 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:56.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:56 smithi155 ceph-mon[33846]: from='client.36152 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-81636-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:56.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:56 smithi155 ceph-mon[33846]: osdmap e216: 8 total, 8 up, 8 in 2024-03-21T12:56:56.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:56 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/739869095' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi043-81636-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:56:56.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:56 smithi155 ceph-mon[33846]: from='client.36152 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi043-81636-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:56:56.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:56 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:56:56.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:56 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-25", "mode": "writeback"}]': finished 2024-03-21T12:56:56.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:56 smithi043 ceph-mon[30062]: from='client.26140 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:56.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:56 smithi043 ceph-mon[30062]: from='client.36152 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-81636-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:56.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:56 smithi043 ceph-mon[30062]: osdmap e216: 8 total, 8 up, 8 in 2024-03-21T12:56:56.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:56 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/739869095' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi043-81636-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:56:56.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:56 smithi043 ceph-mon[30062]: from='client.36152 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi043-81636-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:56:56.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:56 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:56:56.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:56 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-25", "mode": "writeback"}]': finished 2024-03-21T12:56:56.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:56 smithi043 ceph-mon[41699]: from='client.26140 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi043-81661-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:56:56.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:56 smithi043 ceph-mon[41699]: from='client.36152 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi043-81636-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:56:56.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:56 smithi043 ceph-mon[41699]: osdmap e216: 8 total, 8 up, 8 in 2024-03-21T12:56:56.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:56 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/739869095' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi043-81636-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:56:56.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:56 smithi043 ceph-mon[41699]: from='client.36152 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi043-81636-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:56:57.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:57 smithi155 ceph-mon[33846]: pgmap v286: 516 pgs: 64 unknown, 452 active+clean; 584 KiB data, 548 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T12:56:57.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:57 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:57.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:57 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:57.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:57 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:56:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:57 smithi043 ceph-mon[30062]: pgmap v286: 516 pgs: 64 unknown, 452 active+clean; 584 KiB data, 548 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T12:56:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:57 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:57 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:57 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:56:57.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:57 smithi043 ceph-mon[41699]: pgmap v286: 516 pgs: 64 unknown, 452 active+clean; 584 KiB data, 548 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T12:56:57.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:57 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:57.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:57 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:56:57.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:57 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:56:58.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:56:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:56:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:56:58.596 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:56:58 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:56:58.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:58 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-25"}]: dispatch 2024-03-21T12:56:58.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:58 smithi155 ceph-mon[33846]: osdmap e217: 8 total, 8 up, 8 in 2024-03-21T12:56:58.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:58 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-25"}]: dispatch 2024-03-21T12:56:58.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:58 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:56:58.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:58 smithi155 ceph-mon[33846]: from='client.36152 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi043-81636-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-81636-35"}]': finished 2024-03-21T12:56:58.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:58 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-25"}]': finished 2024-03-21T12:56:58.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:58 smithi155 ceph-mon[33846]: osdmap e218: 8 total, 8 up, 8 in 2024-03-21T12:56:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:58 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-25"}]: dispatch 2024-03-21T12:56:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:58 smithi043 ceph-mon[30062]: osdmap e217: 8 total, 8 up, 8 in 2024-03-21T12:56:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:58 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-25"}]: dispatch 2024-03-21T12:56:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:58 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:56:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:58 smithi043 ceph-mon[30062]: from='client.36152 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi043-81636-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-81636-35"}]': finished 2024-03-21T12:56:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:58 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-25"}]': finished 2024-03-21T12:56:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:58 smithi043 ceph-mon[30062]: osdmap e218: 8 total, 8 up, 8 in 2024-03-21T12:56:58.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:58 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-25"}]: dispatch 2024-03-21T12:56:58.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:58 smithi043 ceph-mon[41699]: osdmap e217: 8 total, 8 up, 8 in 2024-03-21T12:56:58.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:58 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-25"}]: dispatch 2024-03-21T12:56:58.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:58 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:56:58.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:58 smithi043 ceph-mon[41699]: from='client.36152 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi043-81636-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi043-81636-35"}]': finished 2024-03-21T12:56:58.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:58 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-25"}]': finished 2024-03-21T12:56:58.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:58 smithi043 ceph-mon[41699]: osdmap e218: 8 total, 8 up, 8 in 2024-03-21T12:56:59.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:59 smithi155 ceph-mon[33846]: pgmap v289: 484 pgs: 32 unknown, 452 active+clean; 584 KiB data, 548 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:56:59.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:59 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:56:59.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:59 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:59.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:56:59 smithi155 ceph-mon[33846]: osdmap e219: 8 total, 8 up, 8 in 2024-03-21T12:56:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:59 smithi043 ceph-mon[30062]: pgmap v289: 484 pgs: 32 unknown, 452 active+clean; 584 KiB data, 548 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:56:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:59 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:56:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:59 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:56:59 smithi043 ceph-mon[30062]: osdmap e219: 8 total, 8 up, 8 in 2024-03-21T12:56:59.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:59 smithi043 ceph-mon[41699]: pgmap v289: 484 pgs: 32 unknown, 452 active+clean; 584 KiB data, 548 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:56:59.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:59 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:56:59.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:59 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:56:59.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:56:59 smithi043 ceph-mon[41699]: osdmap e219: 8 total, 8 up, 8 in 2024-03-21T12:57:01.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:01 smithi155 ceph-mon[33846]: pgmap v292: 396 pgs: 8 unknown, 388 active+clean; 584 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T12:57:01.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:01 smithi155 ceph-mon[33846]: osdmap e220: 8 total, 8 up, 8 in 2024-03-21T12:57:01.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:01 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:01.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:01 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/739869095' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:57:01.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:01 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:01.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:01 smithi155 ceph-mon[33846]: from='client.36152 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:57:01.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:01 smithi043 ceph-mon[30062]: pgmap v292: 396 pgs: 8 unknown, 388 active+clean; 584 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T12:57:01.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:01 smithi043 ceph-mon[30062]: osdmap e220: 8 total, 8 up, 8 in 2024-03-21T12:57:01.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:01 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:01.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:01 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/739869095' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:57:01.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:01 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:01.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:01 smithi043 ceph-mon[30062]: from='client.36152 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:57:01.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:01 smithi043 ceph-mon[41699]: pgmap v292: 396 pgs: 8 unknown, 388 active+clean; 584 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T12:57:01.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:01 smithi043 ceph-mon[41699]: osdmap e220: 8 total, 8 up, 8 in 2024-03-21T12:57:01.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:01 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:01.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:01 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/739869095' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:57:01.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:01 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:01.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:01 smithi043 ceph-mon[41699]: from='client.36152 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:57:02.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:02 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:02.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:02 smithi155 ceph-mon[33846]: from='client.36152 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-81636-35"}]': finished 2024-03-21T12:57:02.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:02 smithi155 ceph-mon[33846]: osdmap e221: 8 total, 8 up, 8 in 2024-03-21T12:57:02.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:02 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/739869095' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:57:02.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:02 smithi155 ceph-mon[33846]: from='client.36152 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:57:02.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:02 smithi155 ceph-mon[33846]: from='client.36152 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-81636-35"}]': finished 2024-03-21T12:57:02.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:02 smithi155 ceph-mon[33846]: osdmap e222: 8 total, 8 up, 8 in 2024-03-21T12:57:02.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:02 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:02.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:02 smithi043 ceph-mon[30062]: from='client.36152 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-81636-35"}]': finished 2024-03-21T12:57:02.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:02 smithi043 ceph-mon[30062]: osdmap e221: 8 total, 8 up, 8 in 2024-03-21T12:57:02.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:02 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/739869095' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:57:02.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:02 smithi043 ceph-mon[30062]: from='client.36152 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:57:02.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:02 smithi043 ceph-mon[30062]: from='client.36152 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-81636-35"}]': finished 2024-03-21T12:57:02.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:02 smithi043 ceph-mon[30062]: osdmap e222: 8 total, 8 up, 8 in 2024-03-21T12:57:02.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:02 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:02.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:02 smithi043 ceph-mon[41699]: from='client.36152 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi043-81636-35"}]': finished 2024-03-21T12:57:02.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:02 smithi043 ceph-mon[41699]: osdmap e221: 8 total, 8 up, 8 in 2024-03-21T12:57:02.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:02 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/739869095' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:57:02.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:02 smithi043 ceph-mon[41699]: from='client.36152 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-81636-35"}]: dispatch 2024-03-21T12:57:02.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:02 smithi043 ceph-mon[41699]: from='client.36152 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi043-81636-35"}]': finished 2024-03-21T12:57:02.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:02 smithi043 ceph-mon[41699]: osdmap e222: 8 total, 8 up, 8 in 2024-03-21T12:57:03.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:03 smithi155 ceph-mon[33846]: pgmap v295: 356 pgs: 32 unknown, 324 active+clean; 584 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:57:03.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:03 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/754954117' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-81661-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:03.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:03 smithi155 ceph-mon[33846]: from='client.26152 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-81661-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:03.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:03 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:03.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:03 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:03.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:03 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3686682818' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:03.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:03 smithi155 ceph-mon[33846]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:03.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:03 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3686682818' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:03.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:03 smithi155 ceph-mon[33846]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:03.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:03 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3686682818' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-81636-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:03.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:03 smithi155 ceph-mon[33846]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-81636-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:03.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:03 smithi155 ceph-mon[33846]: from='client.26152 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-81661-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:03.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:03 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-27", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:57:03.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:03 smithi155 ceph-mon[33846]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-81636-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:03.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:03 smithi155 ceph-mon[33846]: osdmap e223: 8 total, 8 up, 8 in 2024-03-21T12:57:03.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:03 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-27"}]: dispatch 2024-03-21T12:57:03.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:03 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3686682818' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-81636-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:03.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:03 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-27"}]: dispatch 2024-03-21T12:57:03.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:03 smithi155 ceph-mon[33846]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-81636-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:03.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[30062]: pgmap v295: 356 pgs: 32 unknown, 324 active+clean; 584 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:57:03.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/754954117' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-81661-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:03.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[30062]: from='client.26152 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-81661-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:03.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:03.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:03.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3686682818' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:03.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[30062]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:03.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3686682818' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[30062]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3686682818' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-81636-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[30062]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-81636-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[30062]: from='client.26152 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-81661-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-27", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:57:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[30062]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-81636-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[30062]: osdmap e223: 8 total, 8 up, 8 in 2024-03-21T12:57:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-27"}]: dispatch 2024-03-21T12:57:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3686682818' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-81636-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-27"}]: dispatch 2024-03-21T12:57:03.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[30062]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-81636-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:03.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[41699]: pgmap v295: 356 pgs: 32 unknown, 324 active+clean; 584 KiB data, 553 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:57:03.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/754954117' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-81661-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:03.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[41699]: from='client.26152 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-81661-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:03.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:03.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:03.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3686682818' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:03.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[41699]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:03.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3686682818' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:03.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[41699]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:03.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3686682818' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-81636-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:03.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[41699]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-81636-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:03.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[41699]: from='client.26152 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi043-81661-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:03.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-27", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:57:03.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[41699]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi043-81636-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:03.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[41699]: osdmap e223: 8 total, 8 up, 8 in 2024-03-21T12:57:03.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-27"}]: dispatch 2024-03-21T12:57:03.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3686682818' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-81636-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:03.738 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-27"}]: dispatch 2024-03-21T12:57:03.739 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:03 smithi043 ceph-mon[41699]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-81636-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:05.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:04 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:57:05.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:04 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:57:05.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:04 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-27"}]': finished 2024-03-21T12:57:05.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-27", "mode": "writeback"}]: dispatch 2024-03-21T12:57:05.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:04 smithi155 ceph-mon[33846]: osdmap e224: 8 total, 8 up, 8 in 2024-03-21T12:57:05.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:04 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-27", "mode": "writeback"}]: dispatch 2024-03-21T12:57:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:04 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:57:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:04 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:57:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:04 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-27"}]': finished 2024-03-21T12:57:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-27", "mode": "writeback"}]: dispatch 2024-03-21T12:57:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:04 smithi043 ceph-mon[30062]: osdmap e224: 8 total, 8 up, 8 in 2024-03-21T12:57:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:04 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-27", "mode": "writeback"}]: dispatch 2024-03-21T12:57:05.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:04 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:57:05.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:04 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:57:05.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:04 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-27"}]': finished 2024-03-21T12:57:05.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-27", "mode": "writeback"}]: dispatch 2024-03-21T12:57:05.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:04 smithi043 ceph-mon[41699]: osdmap e224: 8 total, 8 up, 8 in 2024-03-21T12:57:05.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:04 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-27", "mode": "writeback"}]: dispatch 2024-03-21T12:57:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:05 smithi155 ceph-mon[33846]: pgmap v298: 388 pgs: 32 creating+peering, 356 active+clean; 584 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 1 op/s 2024-03-21T12:57:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:05 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:57:06.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:05 smithi155 ceph-mon[33846]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-81636-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-81636-36"}]': finished 2024-03-21T12:57:06.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:05 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-27", "mode": "writeback"}]': finished 2024-03-21T12:57:06.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:05 smithi155 ceph-mon[33846]: osdmap e225: 8 total, 8 up, 8 in 2024-03-21T12:57:06.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:05 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2379268674' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-81661-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:06.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:05 smithi155 ceph-mon[33846]: from='client.36170 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-81661-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:06.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:05 smithi043 ceph-mon[30062]: pgmap v298: 388 pgs: 32 creating+peering, 356 active+clean; 584 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 1 op/s 2024-03-21T12:57:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:05 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:57:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:05 smithi043 ceph-mon[30062]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-81636-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-81636-36"}]': finished 2024-03-21T12:57:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:05 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-27", "mode": "writeback"}]': finished 2024-03-21T12:57:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:05 smithi043 ceph-mon[30062]: osdmap e225: 8 total, 8 up, 8 in 2024-03-21T12:57:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:05 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2379268674' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-81661-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:05 smithi043 ceph-mon[30062]: from='client.36170 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-81661-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:05 smithi043 ceph-mon[41699]: pgmap v298: 388 pgs: 32 creating+peering, 356 active+clean; 584 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 1 op/s 2024-03-21T12:57:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:05 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:57:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:05 smithi043 ceph-mon[41699]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi043-81636-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi043-81636-36"}]': finished 2024-03-21T12:57:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:05 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-27", "mode": "writeback"}]': finished 2024-03-21T12:57:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:05 smithi043 ceph-mon[41699]: osdmap e225: 8 total, 8 up, 8 in 2024-03-21T12:57:06.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:05 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2379268674' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-81661-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:06.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:05 smithi043 ceph-mon[41699]: from='client.36170 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-81661-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:07.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:07 smithi155 ceph-mon[33846]: pgmap v301: 396 pgs: 40 unknown, 356 active+clean; 584 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T12:57:07.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:07 smithi155 ceph-mon[33846]: from='client.36170 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-81661-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:07.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:07 smithi155 ceph-mon[33846]: osdmap e226: 8 total, 8 up, 8 in 2024-03-21T12:57:07.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:07 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:07.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:07 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:07.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:07 smithi155 ceph-mon[33846]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:07 smithi043 ceph-mon[30062]: pgmap v301: 396 pgs: 40 unknown, 356 active+clean; 584 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T12:57:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:07 smithi043 ceph-mon[30062]: from='client.36170 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-81661-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:07 smithi043 ceph-mon[30062]: osdmap e226: 8 total, 8 up, 8 in 2024-03-21T12:57:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:07 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:07 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:07 smithi043 ceph-mon[30062]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:07.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:07 smithi043 ceph-mon[41699]: pgmap v301: 396 pgs: 40 unknown, 356 active+clean; 584 KiB data, 558 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T12:57:07.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:07 smithi043 ceph-mon[41699]: from='client.36170 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi043-81661-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:07.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:07 smithi043 ceph-mon[41699]: osdmap e226: 8 total, 8 up, 8 in 2024-03-21T12:57:07.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:07 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:07.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:07 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:07.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:07 smithi043 ceph-mon[41699]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:08.366 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:57:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:57:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:57:08.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:57:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:57:08.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:08 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:57:08.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:08 smithi155 ceph-mon[33846]: osdmap e227: 8 total, 8 up, 8 in 2024-03-21T12:57:08.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:08 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-27"}]: dispatch 2024-03-21T12:57:08.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:08 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-27"}]: dispatch 2024-03-21T12:57:08.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:08 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3686682818' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:08.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:08 smithi155 ceph-mon[33846]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:08.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:08 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1486222997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:08.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:08 smithi155 ceph-mon[33846]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:08.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:08 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1486222997' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:08.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:08 smithi155 ceph-mon[33846]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:08.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:08 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1486222997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81661-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:08.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:08 smithi155 ceph-mon[33846]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81661-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:08.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:08 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:57:08.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:08 smithi043 ceph-mon[30062]: osdmap e227: 8 total, 8 up, 8 in 2024-03-21T12:57:08.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:08 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-27"}]: dispatch 2024-03-21T12:57:08.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:08 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-27"}]: dispatch 2024-03-21T12:57:08.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:08 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3686682818' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:08.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:08 smithi043 ceph-mon[30062]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:08.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:08 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1486222997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:08.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:08 smithi043 ceph-mon[30062]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:08.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:08 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1486222997' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:08.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:08 smithi043 ceph-mon[30062]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:08.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:08 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1486222997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81661-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:08.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:08 smithi043 ceph-mon[30062]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81661-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:08.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:08 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:57:08.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:08 smithi043 ceph-mon[41699]: osdmap e227: 8 total, 8 up, 8 in 2024-03-21T12:57:08.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:08 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-27"}]: dispatch 2024-03-21T12:57:08.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:08 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-27"}]: dispatch 2024-03-21T12:57:08.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:08 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3686682818' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:08.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:08 smithi043 ceph-mon[41699]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:08.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:08 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1486222997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:08.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:08 smithi043 ceph-mon[41699]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:08.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:08 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1486222997' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:08.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:08 smithi043 ceph-mon[41699]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:08.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:08 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1486222997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81661-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:08.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:08 smithi043 ceph-mon[41699]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81661-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:09.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:09 smithi043 ceph-mon[30062]: pgmap v304: 356 pgs: 356 active+clean; 584 KiB data, 558 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:57:09.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:09 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:57:09.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:09 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:57:09.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:09 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-27"}]': finished 2024-03-21T12:57:09.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:09 smithi043 ceph-mon[30062]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81636-36"}]': finished 2024-03-21T12:57:09.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:09 smithi043 ceph-mon[30062]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81661-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:09.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:09 smithi043 ceph-mon[30062]: osdmap e228: 8 total, 8 up, 8 in 2024-03-21T12:57:09.654 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:09 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3686682818' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:09.657 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:09 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1486222997' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81661-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:09.657 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:09 smithi043 ceph-mon[30062]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:09.658 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:09 smithi043 ceph-mon[30062]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81661-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:09.658 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:09 smithi043 ceph-mon[41699]: pgmap v304: 356 pgs: 356 active+clean; 584 KiB data, 558 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:57:09.658 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:09 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:57:09.658 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:09 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:57:09.658 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:09 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-27"}]': finished 2024-03-21T12:57:09.658 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:09 smithi043 ceph-mon[41699]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81636-36"}]': finished 2024-03-21T12:57:09.658 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:09 smithi043 ceph-mon[41699]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81661-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:09.658 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:09 smithi043 ceph-mon[41699]: osdmap e228: 8 total, 8 up, 8 in 2024-03-21T12:57:09.658 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:09 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3686682818' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:09.658 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:09 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1486222997' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81661-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:09.658 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:09 smithi043 ceph-mon[41699]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:09.659 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:09 smithi043 ceph-mon[41699]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81661-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:09.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:09 smithi155 ceph-mon[33846]: pgmap v304: 356 pgs: 356 active+clean; 584 KiB data, 558 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:57:09.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:09 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:57:09.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:09 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:57:09.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:09 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-27"}]': finished 2024-03-21T12:57:09.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:09 smithi155 ceph-mon[33846]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi043-81636-36"}]': finished 2024-03-21T12:57:09.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:09 smithi155 ceph-mon[33846]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81661-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:09.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:09 smithi155 ceph-mon[33846]: osdmap e228: 8 total, 8 up, 8 in 2024-03-21T12:57:09.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:09 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3686682818' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:09.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:09 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1486222997' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81661-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:09.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:09 smithi155 ceph-mon[33846]: from='client.36164 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81636-36"}]: dispatch 2024-03-21T12:57:09.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:09 smithi155 ceph-mon[33846]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81661-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:10.631 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:10 smithi043 ceph-mon[41699]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81636-36"}]': finished 2024-03-21T12:57:10.631 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2176694997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:10.631 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:10 smithi043 ceph-mon[41699]: osdmap e229: 8 total, 8 up, 8 in 2024-03-21T12:57:10.631 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:10 smithi043 ceph-mon[41699]: from='client.36182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:10.631 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2176694997' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:10.631 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:10 smithi043 ceph-mon[41699]: from='client.36182 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:10.631 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2176694997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-81636-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:10.631 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:10 smithi043 ceph-mon[41699]: from='client.36182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-81636-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:10.632 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:10 smithi043 ceph-mon[30062]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81636-36"}]': finished 2024-03-21T12:57:10.632 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2176694997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:10.632 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:10 smithi043 ceph-mon[30062]: osdmap e229: 8 total, 8 up, 8 in 2024-03-21T12:57:10.632 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:10 smithi043 ceph-mon[30062]: from='client.36182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:10.632 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2176694997' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:10.632 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:10 smithi043 ceph-mon[30062]: from='client.36182 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:10.632 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2176694997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-81636-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:10.632 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:10 smithi043 ceph-mon[30062]: from='client.36182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-81636-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:10.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:10 smithi155 ceph-mon[33846]: from='client.36164 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi043-81636-36"}]': finished 2024-03-21T12:57:10.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2176694997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:10.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:10 smithi155 ceph-mon[33846]: osdmap e229: 8 total, 8 up, 8 in 2024-03-21T12:57:10.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:10 smithi155 ceph-mon[33846]: from='client.36182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:10.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2176694997' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:10.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:10 smithi155 ceph-mon[33846]: from='client.36182 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:10.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2176694997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-81636-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:10.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:10 smithi155 ceph-mon[33846]: from='client.36182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-81636-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:11 smithi155 ceph-mon[33846]: pgmap v307: 324 pgs: 324 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:57:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:11 smithi155 ceph-mon[33846]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81661-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81661-46"}]': finished 2024-03-21T12:57:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:11 smithi155 ceph-mon[33846]: from='client.36182 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-81636-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:11 smithi155 ceph-mon[33846]: osdmap e230: 8 total, 8 up, 8 in 2024-03-21T12:57:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:11 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2176694997' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-81636-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:11 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:11.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:11 smithi155 ceph-mon[33846]: from='client.36182 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-81636-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:11.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:11 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:11.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:11 smithi043 ceph-mon[30062]: pgmap v307: 324 pgs: 324 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:57:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:11 smithi043 ceph-mon[30062]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81661-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81661-46"}]': finished 2024-03-21T12:57:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:11 smithi043 ceph-mon[30062]: from='client.36182 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-81636-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:11 smithi043 ceph-mon[30062]: osdmap e230: 8 total, 8 up, 8 in 2024-03-21T12:57:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:11 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2176694997' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-81636-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:11 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:11 smithi043 ceph-mon[30062]: from='client.36182 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-81636-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:11 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:11.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:11 smithi043 ceph-mon[41699]: pgmap v307: 324 pgs: 324 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:57:11.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:11 smithi043 ceph-mon[41699]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81661-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81661-46"}]': finished 2024-03-21T12:57:11.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:11 smithi043 ceph-mon[41699]: from='client.36182 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi043-81636-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:11.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:11 smithi043 ceph-mon[41699]: osdmap e230: 8 total, 8 up, 8 in 2024-03-21T12:57:11.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:11 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2176694997' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-81636-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:11.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:11 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:11.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:11 smithi043 ceph-mon[41699]: from='client.36182 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-81636-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:11.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:11 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:12 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:12 smithi155 ceph-mon[33846]: osdmap e231: 8 total, 8 up, 8 in 2024-03-21T12:57:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:12 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:12 smithi155 ceph-mon[33846]: from='client.36182 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-81636-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-81636-37"}]': finished 2024-03-21T12:57:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:12 smithi155 ceph-mon[33846]: osdmap e232: 8 total, 8 up, 8 in 2024-03-21T12:57:12.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:12 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1486222997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:12.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:12 smithi155 ceph-mon[33846]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:12 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:12 smithi043 ceph-mon[30062]: osdmap e231: 8 total, 8 up, 8 in 2024-03-21T12:57:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:12 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:12 smithi043 ceph-mon[30062]: from='client.36182 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-81636-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-81636-37"}]': finished 2024-03-21T12:57:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:12 smithi043 ceph-mon[30062]: osdmap e232: 8 total, 8 up, 8 in 2024-03-21T12:57:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:12 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1486222997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:12 smithi043 ceph-mon[30062]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:12.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:12 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:12.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:12 smithi043 ceph-mon[41699]: osdmap e231: 8 total, 8 up, 8 in 2024-03-21T12:57:12.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:12 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:12.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:12 smithi043 ceph-mon[41699]: from='client.36182 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi043-81636-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi043-81636-37"}]': finished 2024-03-21T12:57:12.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:12 smithi043 ceph-mon[41699]: osdmap e232: 8 total, 8 up, 8 in 2024-03-21T12:57:12.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:12 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1486222997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:12.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:12 smithi043 ceph-mon[41699]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:13 smithi155 ceph-mon[33846]: pgmap v310: 364 pgs: 40 unknown, 324 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:57:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:13 smithi155 ceph-mon[33846]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81661-46"}]': finished 2024-03-21T12:57:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:13 smithi155 ceph-mon[33846]: osdmap e233: 8 total, 8 up, 8 in 2024-03-21T12:57:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:13 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1486222997' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:13 smithi155 ceph-mon[33846]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:13 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:13 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:13 smithi043 ceph-mon[30062]: pgmap v310: 364 pgs: 40 unknown, 324 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:57:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:13 smithi043 ceph-mon[30062]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81661-46"}]': finished 2024-03-21T12:57:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:13 smithi043 ceph-mon[30062]: osdmap e233: 8 total, 8 up, 8 in 2024-03-21T12:57:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:13 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1486222997' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:13 smithi043 ceph-mon[30062]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:13 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:13 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:13 smithi043 ceph-mon[41699]: pgmap v310: 364 pgs: 40 unknown, 324 active+clean; 584 KiB data, 563 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:57:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:13 smithi043 ceph-mon[41699]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81661-46"}]': finished 2024-03-21T12:57:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:13 smithi043 ceph-mon[41699]: osdmap e233: 8 total, 8 up, 8 in 2024-03-21T12:57:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:13 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1486222997' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:13 smithi043 ceph-mon[41699]: from='client.36176 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81661-46"}]: dispatch 2024-03-21T12:57:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:13 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:13 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:15.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:15 smithi155 ceph-mon[33846]: pgmap v313: 364 pgs: 8 creating+peering, 356 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-21T12:57:15.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:15 smithi155 ceph-mon[33846]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81661-46"}]': finished 2024-03-21T12:57:15.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:15 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-29", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:57:15.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:15 smithi155 ceph-mon[33846]: osdmap e234: 8 total, 8 up, 8 in 2024-03-21T12:57:15.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:15 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-29"}]: dispatch 2024-03-21T12:57:15.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:15 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-29"}]: dispatch 2024-03-21T12:57:15.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:15 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2176694997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:15.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:15 smithi155 ceph-mon[33846]: from='client.36182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:15.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:15 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81661-47"}]: dispatch 2024-03-21T12:57:15.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:15 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81661-47"}]: dispatch 2024-03-21T12:57:15.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:15 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81661-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:15.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:15 smithi043 ceph-mon[41699]: pgmap v313: 364 pgs: 8 creating+peering, 356 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-21T12:57:15.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:15 smithi043 ceph-mon[41699]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81661-46"}]': finished 2024-03-21T12:57:15.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:15 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-29", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:57:15.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:15 smithi043 ceph-mon[41699]: osdmap e234: 8 total, 8 up, 8 in 2024-03-21T12:57:15.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:15 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-29"}]: dispatch 2024-03-21T12:57:15.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:15 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-29"}]: dispatch 2024-03-21T12:57:15.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:15 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2176694997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:15.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:15 smithi043 ceph-mon[41699]: from='client.36182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:15.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:15 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81661-47"}]: dispatch 2024-03-21T12:57:15.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:15 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81661-47"}]: dispatch 2024-03-21T12:57:15.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:15 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81661-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:15.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:15 smithi043 ceph-mon[30062]: pgmap v313: 364 pgs: 8 creating+peering, 356 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-21T12:57:15.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:15 smithi043 ceph-mon[30062]: from='client.36176 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81661-46"}]': finished 2024-03-21T12:57:15.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:15 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-29", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:57:15.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:15 smithi043 ceph-mon[30062]: osdmap e234: 8 total, 8 up, 8 in 2024-03-21T12:57:15.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:15 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-29"}]: dispatch 2024-03-21T12:57:15.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:15 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-29"}]: dispatch 2024-03-21T12:57:15.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:15 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2176694997' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:15.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:15 smithi043 ceph-mon[30062]: from='client.36182 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:15.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:15 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81661-47"}]: dispatch 2024-03-21T12:57:15.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:15 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81661-47"}]: dispatch 2024-03-21T12:57:15.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:15 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81661-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:16.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:16 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-29"}]': finished 2024-03-21T12:57:16.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:16 smithi155 ceph-mon[33846]: from='client.36182 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-81636-37"}]': finished 2024-03-21T12:57:16.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:16 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81661-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:16.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:16 smithi155 ceph-mon[33846]: osdmap e235: 8 total, 8 up, 8 in 2024-03-21T12:57:16.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:16 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81661-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81661-47"}]: dispatch 2024-03-21T12:57:16.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:16 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-29", "mode": "writeback"}]: dispatch 2024-03-21T12:57:16.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:16 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2176694997' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:16.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:16 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-29", "mode": "writeback"}]: dispatch 2024-03-21T12:57:16.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:16 smithi155 ceph-mon[33846]: from='client.36182 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:16.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:16 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:57:16.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:16 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-29", "mode": "writeback"}]': finished 2024-03-21T12:57:16.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:16 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-29"}]': finished 2024-03-21T12:57:16.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:16 smithi043 ceph-mon[30062]: from='client.36182 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-81636-37"}]': finished 2024-03-21T12:57:16.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:16 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81661-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:16.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:16 smithi043 ceph-mon[30062]: osdmap e235: 8 total, 8 up, 8 in 2024-03-21T12:57:16.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:16 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81661-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81661-47"}]: dispatch 2024-03-21T12:57:16.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:16 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-29", "mode": "writeback"}]: dispatch 2024-03-21T12:57:16.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:16 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2176694997' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:16.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:16 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-29", "mode": "writeback"}]: dispatch 2024-03-21T12:57:16.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:16 smithi043 ceph-mon[30062]: from='client.36182 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:16.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:16 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:57:16.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:16 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-29", "mode": "writeback"}]': finished 2024-03-21T12:57:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:16 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-29"}]': finished 2024-03-21T12:57:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:16 smithi043 ceph-mon[41699]: from='client.36182 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi043-81636-37"}]': finished 2024-03-21T12:57:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:16 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi043-81661-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:16 smithi043 ceph-mon[41699]: osdmap e235: 8 total, 8 up, 8 in 2024-03-21T12:57:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:16 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81661-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81661-47"}]: dispatch 2024-03-21T12:57:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:16 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-29", "mode": "writeback"}]: dispatch 2024-03-21T12:57:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:16 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2176694997' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:16 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-29", "mode": "writeback"}]: dispatch 2024-03-21T12:57:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:16 smithi043 ceph-mon[41699]: from='client.36182 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-81636-37"}]: dispatch 2024-03-21T12:57:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:16 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:57:16.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:16 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-29", "mode": "writeback"}]': finished 2024-03-21T12:57:17.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:17 smithi155 ceph-mon[33846]: pgmap v316: 356 pgs: 356 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-21T12:57:17.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:17 smithi155 ceph-mon[33846]: from='client.36182 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-81636-37"}]': finished 2024-03-21T12:57:17.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:17 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/815140742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:17 smithi155 ceph-mon[33846]: osdmap e236: 8 total, 8 up, 8 in 2024-03-21T12:57:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:17 smithi155 ceph-mon[33846]: from='client.36194 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:17 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/815140742' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:17 smithi155 ceph-mon[33846]: from='client.36194 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:17 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/815140742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-81636-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:17 smithi155 ceph-mon[33846]: from='client.36194 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-81636-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.17", "id": [6, 7]}]: dispatch 2024-03-21T12:57:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.15", "id": [5, 0]}]: dispatch 2024-03-21T12:57:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1f", "id": [5, 1]}]: dispatch 2024-03-21T12:57:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:17 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.17", "id": [6, 7]}]: dispatch 2024-03-21T12:57:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:17 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.15", "id": [5, 0]}]: dispatch 2024-03-21T12:57:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:17 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1f", "id": [5, 1]}]: dispatch 2024-03-21T12:57:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:17 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81661-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81661-47"}]': finished 2024-03-21T12:57:17.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:17 smithi155 ceph-mon[33846]: from='client.36194 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-81636-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:17.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:17 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.17", "id": [6, 7]}]': finished 2024-03-21T12:57:17.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:17 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.15", "id": [5, 0]}]': finished 2024-03-21T12:57:17.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:17 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1f", "id": [5, 1]}]': finished 2024-03-21T12:57:17.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:17 smithi155 ceph-mon[33846]: osdmap e237: 8 total, 8 up, 8 in 2024-03-21T12:57:17.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:17 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/815140742' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-81636-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:17.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:17 smithi155 ceph-mon[33846]: from='client.36194 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-81636-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[30062]: pgmap v316: 356 pgs: 356 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-21T12:57:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[30062]: from='client.36182 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-81636-37"}]': finished 2024-03-21T12:57:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/815140742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:17.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[30062]: osdmap e236: 8 total, 8 up, 8 in 2024-03-21T12:57:17.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[30062]: from='client.36194 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:17.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/815140742' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:17.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[30062]: from='client.36194 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:17.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/815140742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-81636-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:17.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[30062]: from='client.36194 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-81636-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:17.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.17", "id": [6, 7]}]: dispatch 2024-03-21T12:57:17.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.15", "id": [5, 0]}]: dispatch 2024-03-21T12:57:17.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1f", "id": [5, 1]}]: dispatch 2024-03-21T12:57:17.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.17", "id": [6, 7]}]: dispatch 2024-03-21T12:57:17.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.15", "id": [5, 0]}]: dispatch 2024-03-21T12:57:17.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1f", "id": [5, 1]}]: dispatch 2024-03-21T12:57:17.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81661-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81661-47"}]': finished 2024-03-21T12:57:17.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[30062]: from='client.36194 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-81636-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:17.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.17", "id": [6, 7]}]': finished 2024-03-21T12:57:17.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.15", "id": [5, 0]}]': finished 2024-03-21T12:57:17.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1f", "id": [5, 1]}]': finished 2024-03-21T12:57:17.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[30062]: osdmap e237: 8 total, 8 up, 8 in 2024-03-21T12:57:17.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/815140742' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-81636-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:17.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[30062]: from='client.36194 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-81636-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:17.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[41699]: pgmap v316: 356 pgs: 356 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-21T12:57:17.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[41699]: from='client.36182 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi043-81636-37"}]': finished 2024-03-21T12:57:17.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/815140742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:17.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[41699]: osdmap e236: 8 total, 8 up, 8 in 2024-03-21T12:57:17.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[41699]: from='client.36194 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:17.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/815140742' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:17.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[41699]: from='client.36194 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:17.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/815140742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-81636-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:17.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[41699]: from='client.36194 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-81636-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:17.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.17", "id": [6, 7]}]: dispatch 2024-03-21T12:57:17.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.15", "id": [5, 0]}]: dispatch 2024-03-21T12:57:17.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1f", "id": [5, 1]}]: dispatch 2024-03-21T12:57:17.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.17", "id": [6, 7]}]: dispatch 2024-03-21T12:57:17.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.15", "id": [5, 0]}]: dispatch 2024-03-21T12:57:17.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1f", "id": [5, 1]}]: dispatch 2024-03-21T12:57:17.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi043-81661-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi043-81661-47"}]': finished 2024-03-21T12:57:17.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[41699]: from='client.36194 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi043-81636-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:17.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "102.17", "id": [6, 7]}]': finished 2024-03-21T12:57:17.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.15", "id": [5, 0]}]': finished 2024-03-21T12:57:17.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "242.1f", "id": [5, 1]}]': finished 2024-03-21T12:57:17.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[41699]: osdmap e237: 8 total, 8 up, 8 in 2024-03-21T12:57:17.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/815140742' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-81636-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:17.988 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:17 smithi043 ceph-mon[41699]: from='client.36194 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-81636-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:18.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:57:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:57:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:57:18.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:57:18 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:57:19.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:19 smithi155 ceph-mon[33846]: pgmap v319: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:57:19.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:19 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:57:19.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:19 smithi155 ceph-mon[33846]: osdmap e238: 8 total, 8 up, 8 in 2024-03-21T12:57:19.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:19 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:19.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:19 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:19 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:19 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:57:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:19 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:57:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:19 smithi043 ceph-mon[30062]: pgmap v319: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:57:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:19 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:57:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:19 smithi043 ceph-mon[30062]: osdmap e238: 8 total, 8 up, 8 in 2024-03-21T12:57:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:19 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:19 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:19 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:19 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:57:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:19 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:57:19.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:19 smithi043 ceph-mon[41699]: pgmap v319: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:57:19.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:19 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:57:19.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:19 smithi043 ceph-mon[41699]: osdmap e238: 8 total, 8 up, 8 in 2024-03-21T12:57:19.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:19 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:19 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:19 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:19 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:57:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:19 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:57:20.691 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:20 smithi043 ceph-mon[30062]: from='client.36194 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-81636-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-81636-38"}]': finished 2024-03-21T12:57:20.691 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:20 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:57:20.691 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:20 smithi043 ceph-mon[30062]: osdmap e239: 8 total, 8 up, 8 in 2024-03-21T12:57:20.691 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:20 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-29"}]: dispatch 2024-03-21T12:57:20.691 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:20 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-29"}]: dispatch 2024-03-21T12:57:20.691 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:20 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81661-47"}]: dispatch 2024-03-21T12:57:20.692 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:20 smithi043 ceph-mon[41699]: from='client.36194 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-81636-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-81636-38"}]': finished 2024-03-21T12:57:20.692 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:20 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:57:20.692 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:20 smithi043 ceph-mon[41699]: osdmap e239: 8 total, 8 up, 8 in 2024-03-21T12:57:20.692 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:20 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-29"}]: dispatch 2024-03-21T12:57:20.692 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:20 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-29"}]: dispatch 2024-03-21T12:57:20.692 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:20 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81661-47"}]: dispatch 2024-03-21T12:57:20.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:20 smithi155 ceph-mon[33846]: from='client.36194 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi043-81636-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi043-81636-38"}]': finished 2024-03-21T12:57:20.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:20 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:57:20.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:20 smithi155 ceph-mon[33846]: osdmap e239: 8 total, 8 up, 8 in 2024-03-21T12:57:20.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:20 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-29"}]: dispatch 2024-03-21T12:57:20.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:20 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-29"}]: dispatch 2024-03-21T12:57:20.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:20 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81661-47"}]: dispatch 2024-03-21T12:57:21.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:21 smithi155 ceph-mon[33846]: pgmap v322: 364 pgs: 8 unknown, 3 peering, 353 active+clean; 8.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 255 B/s wr, 26 op/s 2024-03-21T12:57:21.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:21 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:57:21.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:21 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-29"}]': finished 2024-03-21T12:57:21.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:21 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81661-47"}]': finished 2024-03-21T12:57:21.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:21 smithi155 ceph-mon[33846]: osdmap e240: 8 total, 8 up, 8 in 2024-03-21T12:57:21.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:21 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81661-47"}]: dispatch 2024-03-21T12:57:21.733 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:21 smithi043 ceph-mon[41699]: pgmap v322: 364 pgs: 8 unknown, 3 peering, 353 active+clean; 8.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 255 B/s wr, 26 op/s 2024-03-21T12:57:21.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:21 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:57:21.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:21 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-29"}]': finished 2024-03-21T12:57:21.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:21 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81661-47"}]': finished 2024-03-21T12:57:21.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:21 smithi043 ceph-mon[41699]: osdmap e240: 8 total, 8 up, 8 in 2024-03-21T12:57:21.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:21 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81661-47"}]: dispatch 2024-03-21T12:57:21.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:21 smithi043 ceph-mon[30062]: pgmap v322: 364 pgs: 8 unknown, 3 peering, 353 active+clean; 8.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 255 B/s wr, 26 op/s 2024-03-21T12:57:21.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:21 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:57:21.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:21 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-29"}]': finished 2024-03-21T12:57:21.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:21 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi043-81661-47"}]': finished 2024-03-21T12:57:21.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:21 smithi043 ceph-mon[30062]: osdmap e240: 8 total, 8 up, 8 in 2024-03-21T12:57:21.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:21 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81661-47"}]: dispatch 2024-03-21T12:57:22.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:22 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81661-47"}]': finished 2024-03-21T12:57:22.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:22 smithi155 ceph-mon[33846]: osdmap e241: 8 total, 8 up, 8 in 2024-03-21T12:57:22.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:22 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/815140742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:22.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:22 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3472529744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:22.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:22 smithi155 ceph-mon[33846]: from='client.36194 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:22.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:22 smithi155 ceph-mon[33846]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:22.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:22 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3472529744' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:22.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:22 smithi155 ceph-mon[33846]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:22.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:22 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3472529744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-81661-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:22.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:22 smithi155 ceph-mon[33846]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-81661-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:22.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:22 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81661-47"}]': finished 2024-03-21T12:57:22.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:22 smithi043 ceph-mon[30062]: osdmap e241: 8 total, 8 up, 8 in 2024-03-21T12:57:22.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:22 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/815140742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:22.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:22 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3472529744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:22.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:22 smithi043 ceph-mon[30062]: from='client.36194 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:22.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:22 smithi043 ceph-mon[30062]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:22.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:22 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3472529744' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:22.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:22 smithi043 ceph-mon[30062]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:22.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:22 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3472529744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-81661-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:22.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:22 smithi043 ceph-mon[30062]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-81661-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:22.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:22 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3153232510' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi043-81661-47"}]': finished 2024-03-21T12:57:22.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:22 smithi043 ceph-mon[41699]: osdmap e241: 8 total, 8 up, 8 in 2024-03-21T12:57:22.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:22 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/815140742' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:22.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:22 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3472529744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:22.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:22 smithi043 ceph-mon[41699]: from='client.36194 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:22.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:22 smithi043 ceph-mon[41699]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:22.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:22 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3472529744' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:22.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:22 smithi043 ceph-mon[41699]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:22.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:22 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3472529744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-81661-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:22.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:22 smithi043 ceph-mon[41699]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-81661-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:23.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:23 smithi155 ceph-mon[33846]: pgmap v325: 324 pgs: 1 peering, 323 active+clean; 4.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:57:23.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:23 smithi155 ceph-mon[33846]: from='client.36194 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81636-38"}]': finished 2024-03-21T12:57:23.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:23 smithi155 ceph-mon[33846]: from='client.36200 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-81661-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:23.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:23 smithi155 ceph-mon[33846]: osdmap e242: 8 total, 8 up, 8 in 2024-03-21T12:57:23.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:23 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3472529744' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-81661-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:23.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:23 smithi155 ceph-mon[33846]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-81661-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:23.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:23 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/815140742' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:23.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:23 smithi155 ceph-mon[33846]: from='client.36194 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:23.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:23 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:23.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:23 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:23 smithi043 ceph-mon[30062]: pgmap v325: 324 pgs: 1 peering, 323 active+clean; 4.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:57:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:23 smithi043 ceph-mon[30062]: from='client.36194 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81636-38"}]': finished 2024-03-21T12:57:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:23 smithi043 ceph-mon[30062]: from='client.36200 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-81661-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:23 smithi043 ceph-mon[30062]: osdmap e242: 8 total, 8 up, 8 in 2024-03-21T12:57:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:23 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3472529744' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-81661-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:23 smithi043 ceph-mon[30062]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-81661-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:23 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/815140742' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:23 smithi043 ceph-mon[30062]: from='client.36194 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:23 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:23 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:23.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:23 smithi043 ceph-mon[41699]: pgmap v325: 324 pgs: 1 peering, 323 active+clean; 4.6 MiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:57:23.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:23 smithi043 ceph-mon[41699]: from='client.36194 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi043-81636-38"}]': finished 2024-03-21T12:57:23.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:23 smithi043 ceph-mon[41699]: from='client.36200 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi043-81661-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:23.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:23 smithi043 ceph-mon[41699]: osdmap e242: 8 total, 8 up, 8 in 2024-03-21T12:57:23.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:23 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3472529744' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-81661-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:23.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:23 smithi043 ceph-mon[41699]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-81661-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:23.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:23 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/815140742' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:23.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:23 smithi043 ceph-mon[41699]: from='client.36194 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81636-38"}]: dispatch 2024-03-21T12:57:23.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:23 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:23.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:23 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:24.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:24 smithi155 ceph-mon[33846]: from='client.36194 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81636-38"}]': finished 2024-03-21T12:57:24.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:24 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:24.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:24 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:24.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:24 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1375598894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:24.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:24 smithi155 ceph-mon[33846]: osdmap e243: 8 total, 8 up, 8 in 2024-03-21T12:57:24.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:24 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:24.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:24 smithi155 ceph-mon[33846]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:24.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:24 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1375598894' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:24.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:24 smithi155 ceph-mon[33846]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:24.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:24 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1375598894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-81636-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:24.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:24 smithi155 ceph-mon[33846]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-81636-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:24.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:24 smithi155 ceph-mon[33846]: from='client.36200 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-81661-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-81661-48"}]': finished 2024-03-21T12:57:24.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:24 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-31", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:57:24.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:24 smithi155 ceph-mon[33846]: from='client.36206 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-81636-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:24.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:24 smithi155 ceph-mon[33846]: osdmap e244: 8 total, 8 up, 8 in 2024-03-21T12:57:24.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:24 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1375598894' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-81636-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:24.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:24 smithi155 ceph-mon[33846]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-81636-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:24.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:24 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-31"}]: dispatch 2024-03-21T12:57:24.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:24 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-31"}]: dispatch 2024-03-21T12:57:24.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[30062]: from='client.36194 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81636-38"}]': finished 2024-03-21T12:57:24.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:24.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:24.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1375598894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:24.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[30062]: osdmap e243: 8 total, 8 up, 8 in 2024-03-21T12:57:24.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:24.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[30062]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:24.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1375598894' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:24.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[30062]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:24.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1375598894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-81636-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:24.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[30062]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-81636-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:24.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[30062]: from='client.36200 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-81661-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-81661-48"}]': finished 2024-03-21T12:57:24.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-31", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:57:24.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[30062]: from='client.36206 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-81636-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:24.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[30062]: osdmap e244: 8 total, 8 up, 8 in 2024-03-21T12:57:24.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1375598894' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-81636-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:24.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[30062]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-81636-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:24.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-31"}]: dispatch 2024-03-21T12:57:24.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-31"}]: dispatch 2024-03-21T12:57:24.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[41699]: from='client.36194 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi043-81636-38"}]': finished 2024-03-21T12:57:24.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:24.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:24.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1375598894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:24.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[41699]: osdmap e243: 8 total, 8 up, 8 in 2024-03-21T12:57:24.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:24.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[41699]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:24.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1375598894' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:24.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[41699]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:24.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1375598894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-81636-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:24.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[41699]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-81636-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:24.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[41699]: from='client.36200 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi043-81661-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi043-81661-48"}]': finished 2024-03-21T12:57:24.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-31", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:57:24.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[41699]: from='client.36206 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi043-81636-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:24.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[41699]: osdmap e244: 8 total, 8 up, 8 in 2024-03-21T12:57:24.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1375598894' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-81636-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:24.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[41699]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-81636-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:24.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-31"}]: dispatch 2024-03-21T12:57:24.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:24 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-31"}]: dispatch 2024-03-21T12:57:25.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:25 smithi155 ceph-mon[33846]: pgmap v328: 356 pgs: 32 creating+peering, 324 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 5 B/s, 2 objects/s recovering 2024-03-21T12:57:25.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:25 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-31"}]': finished 2024-03-21T12:57:25.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:25 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-31", "mode": "writeback"}]: dispatch 2024-03-21T12:57:25.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:25 smithi155 ceph-mon[33846]: osdmap e245: 8 total, 8 up, 8 in 2024-03-21T12:57:25.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:25 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-31", "mode": "writeback"}]: dispatch 2024-03-21T12:57:25.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:25 smithi043 ceph-mon[30062]: pgmap v328: 356 pgs: 32 creating+peering, 324 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 5 B/s, 2 objects/s recovering 2024-03-21T12:57:25.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:25 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-31"}]': finished 2024-03-21T12:57:25.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:25 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-31", "mode": "writeback"}]: dispatch 2024-03-21T12:57:25.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:25 smithi043 ceph-mon[30062]: osdmap e245: 8 total, 8 up, 8 in 2024-03-21T12:57:25.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:25 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-31", "mode": "writeback"}]: dispatch 2024-03-21T12:57:25.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:25 smithi043 ceph-mon[41699]: pgmap v328: 356 pgs: 32 creating+peering, 324 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 5 B/s, 2 objects/s recovering 2024-03-21T12:57:25.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:25 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-31"}]': finished 2024-03-21T12:57:25.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:25 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-31", "mode": "writeback"}]: dispatch 2024-03-21T12:57:25.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:25 smithi043 ceph-mon[41699]: osdmap e245: 8 total, 8 up, 8 in 2024-03-21T12:57:25.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:25 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-31", "mode": "writeback"}]: dispatch 2024-03-21T12:57:26.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:26 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:57:26.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:26 smithi155 ceph-mon[33846]: from='client.36206 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-81636-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-81636-39"}]': finished 2024-03-21T12:57:26.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:26 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-31", "mode": "writeback"}]': finished 2024-03-21T12:57:26.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:26 smithi155 ceph-mon[33846]: osdmap e246: 8 total, 8 up, 8 in 2024-03-21T12:57:26.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:26 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3472529744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:26.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:26 smithi155 ceph-mon[33846]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:26.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:26 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:26.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:26 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:26.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:26 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:57:26.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:26 smithi043 ceph-mon[30062]: from='client.36206 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-81636-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-81636-39"}]': finished 2024-03-21T12:57:26.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:26 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-31", "mode": "writeback"}]': finished 2024-03-21T12:57:26.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:26 smithi043 ceph-mon[30062]: osdmap e246: 8 total, 8 up, 8 in 2024-03-21T12:57:26.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:26 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3472529744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:26.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:26 smithi043 ceph-mon[30062]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:26.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:26 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:26.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:26 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:26.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:26 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:57:26.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:26 smithi043 ceph-mon[41699]: from='client.36206 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi043-81636-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi043-81636-39"}]': finished 2024-03-21T12:57:26.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:26 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-31", "mode": "writeback"}]': finished 2024-03-21T12:57:26.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:26 smithi043 ceph-mon[41699]: osdmap e246: 8 total, 8 up, 8 in 2024-03-21T12:57:26.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:26 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3472529744' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:26.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:26 smithi043 ceph-mon[41699]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:26.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:26 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:26.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:26 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:27.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:27 smithi155 ceph-mon[33846]: pgmap v331: 364 pgs: 8 unknown, 32 creating+peering, 324 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s; 5 B/s, 2 objects/s recovering 2024-03-21T12:57:27.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:27 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:27.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:27 smithi155 ceph-mon[33846]: from='client.36200 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-81661-48"}]': finished 2024-03-21T12:57:27.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:27 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:57:27.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:27 smithi155 ceph-mon[33846]: osdmap e247: 8 total, 8 up, 8 in 2024-03-21T12:57:27.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:27 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3472529744' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:27.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:27 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-31"}]: dispatch 2024-03-21T12:57:27.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:27 smithi155 ceph-mon[33846]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:27.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:27 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-31"}]: dispatch 2024-03-21T12:57:27.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:27 smithi043 ceph-mon[30062]: pgmap v331: 364 pgs: 8 unknown, 32 creating+peering, 324 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s; 5 B/s, 2 objects/s recovering 2024-03-21T12:57:27.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:27 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:27.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:27 smithi043 ceph-mon[30062]: from='client.36200 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-81661-48"}]': finished 2024-03-21T12:57:27.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:27 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:57:27.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:27 smithi043 ceph-mon[30062]: osdmap e247: 8 total, 8 up, 8 in 2024-03-21T12:57:27.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:27 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3472529744' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:27.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:27 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-31"}]: dispatch 2024-03-21T12:57:27.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:27 smithi043 ceph-mon[30062]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:27.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:27 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-31"}]: dispatch 2024-03-21T12:57:27.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:27 smithi043 ceph-mon[41699]: pgmap v331: 364 pgs: 8 unknown, 32 creating+peering, 324 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s; 5 B/s, 2 objects/s recovering 2024-03-21T12:57:27.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:27 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:27.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:27 smithi043 ceph-mon[41699]: from='client.36200 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi043-81661-48"}]': finished 2024-03-21T12:57:27.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:27 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:57:27.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:27 smithi043 ceph-mon[41699]: osdmap e247: 8 total, 8 up, 8 in 2024-03-21T12:57:27.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:27 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3472529744' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:27.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:27 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-31"}]: dispatch 2024-03-21T12:57:27.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:27 smithi043 ceph-mon[41699]: from='client.36200 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-81661-48"}]: dispatch 2024-03-21T12:57:27.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:27 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-31"}]: dispatch 2024-03-21T12:57:28.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:57:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:57:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:57:28.647 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:57:28 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:57:28.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:28 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:57:28.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:28 smithi155 ceph-mon[33846]: from='client.36200 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-81661-48"}]': finished 2024-03-21T12:57:28.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:28 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-31"}]': finished 2024-03-21T12:57:28.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:28 smithi155 ceph-mon[33846]: osdmap e248: 8 total, 8 up, 8 in 2024-03-21T12:57:28.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1375598894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:28.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:28 smithi155 ceph-mon[33846]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:28.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-81661-49"}]: dispatch 2024-03-21T12:57:28.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-81661-49"}]: dispatch 2024-03-21T12:57:28.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-81661-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:28.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:28 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:57:28.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:28 smithi043 ceph-mon[30062]: from='client.36200 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-81661-48"}]': finished 2024-03-21T12:57:28.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:28 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-31"}]': finished 2024-03-21T12:57:28.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:28 smithi043 ceph-mon[30062]: osdmap e248: 8 total, 8 up, 8 in 2024-03-21T12:57:28.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1375598894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:28.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:28 smithi043 ceph-mon[30062]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:28.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-81661-49"}]: dispatch 2024-03-21T12:57:28.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-81661-49"}]: dispatch 2024-03-21T12:57:28.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-81661-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:28.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:28 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:57:28.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:28 smithi043 ceph-mon[41699]: from='client.36200 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi043-81661-48"}]': finished 2024-03-21T12:57:28.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:28 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-31"}]': finished 2024-03-21T12:57:28.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:28 smithi043 ceph-mon[41699]: osdmap e248: 8 total, 8 up, 8 in 2024-03-21T12:57:28.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1375598894' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:28.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:28 smithi043 ceph-mon[41699]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:28.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-81661-49"}]: dispatch 2024-03-21T12:57:28.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-81661-49"}]: dispatch 2024-03-21T12:57:28.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-81661-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:29 smithi155 ceph-mon[33846]: pgmap v334: 364 pgs: 8 unknown, 32 creating+peering, 324 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:57:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:29 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:57:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:29 smithi155 ceph-mon[33846]: from='client.36206 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-81636-39"}]': finished 2024-03-21T12:57:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:29 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-81661-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:29 smithi155 ceph-mon[33846]: osdmap e249: 8 total, 8 up, 8 in 2024-03-21T12:57:29.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:29 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-81661-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-81661-49"}]: dispatch 2024-03-21T12:57:29.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:29 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1375598894' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:29.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:29 smithi155 ceph-mon[33846]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:29 smithi043 ceph-mon[30062]: pgmap v334: 364 pgs: 8 unknown, 32 creating+peering, 324 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:57:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:29 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:57:29.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:29 smithi043 ceph-mon[30062]: from='client.36206 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-81636-39"}]': finished 2024-03-21T12:57:29.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:29 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-81661-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:29.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:29 smithi043 ceph-mon[30062]: osdmap e249: 8 total, 8 up, 8 in 2024-03-21T12:57:29.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:29 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-81661-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-81661-49"}]: dispatch 2024-03-21T12:57:29.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:29 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1375598894' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:29.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:29 smithi043 ceph-mon[30062]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:29.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:29 smithi043 ceph-mon[41699]: pgmap v334: 364 pgs: 8 unknown, 32 creating+peering, 324 active+clean; 4.6 MiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:57:29.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:29 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:57:29.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:29 smithi043 ceph-mon[41699]: from='client.36206 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi043-81636-39"}]': finished 2024-03-21T12:57:29.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:29 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi043-81661-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:29.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:29 smithi043 ceph-mon[41699]: osdmap e249: 8 total, 8 up, 8 in 2024-03-21T12:57:29.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:29 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-81661-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-81661-49"}]: dispatch 2024-03-21T12:57:29.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:29 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1375598894' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:29.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:29 smithi043 ceph-mon[41699]: from='client.36206 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-81636-39"}]: dispatch 2024-03-21T12:57:31.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:31 smithi155 ceph-mon[33846]: pgmap v337: 324 pgs: 324 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:57:31.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:31 smithi155 ceph-mon[33846]: from='client.36206 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-81636-39"}]': finished 2024-03-21T12:57:31.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:31 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3257114091' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:31.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:31 smithi155 ceph-mon[33846]: osdmap e250: 8 total, 8 up, 8 in 2024-03-21T12:57:31.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:31 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:31.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:31 smithi155 ceph-mon[33846]: from='client.36218 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:31.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:31 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:31.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:31 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3257114091' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:31.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:31 smithi155 ceph-mon[33846]: from='client.36218 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:31.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:31 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3257114091' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-81636-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:31.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:31 smithi155 ceph-mon[33846]: from='client.36218 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-81636-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:31.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:31 smithi043 ceph-mon[30062]: pgmap v337: 324 pgs: 324 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:57:31.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:31 smithi043 ceph-mon[30062]: from='client.36206 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-81636-39"}]': finished 2024-03-21T12:57:31.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:31 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3257114091' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:31.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:31 smithi043 ceph-mon[30062]: osdmap e250: 8 total, 8 up, 8 in 2024-03-21T12:57:31.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:31 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:31.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:31 smithi043 ceph-mon[30062]: from='client.36218 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:31.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:31 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:31.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:31 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3257114091' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:31.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:31 smithi043 ceph-mon[30062]: from='client.36218 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:31.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:31 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3257114091' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-81636-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:31.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:31 smithi043 ceph-mon[30062]: from='client.36218 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-81636-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:31.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:31 smithi043 ceph-mon[41699]: pgmap v337: 324 pgs: 324 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:57:31.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:31 smithi043 ceph-mon[41699]: from='client.36206 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi043-81636-39"}]': finished 2024-03-21T12:57:31.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:31 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3257114091' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:31.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:31 smithi043 ceph-mon[41699]: osdmap e250: 8 total, 8 up, 8 in 2024-03-21T12:57:31.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:31 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:31.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:31 smithi043 ceph-mon[41699]: from='client.36218 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:31.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:31 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:31.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:31 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3257114091' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:31.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:31 smithi043 ceph-mon[41699]: from='client.36218 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:31.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:31 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3257114091' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-81636-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:31.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:31 smithi043 ceph-mon[41699]: from='client.36218 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-81636-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:32.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:32 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-81661-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-81661-49"}]': finished 2024-03-21T12:57:32.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:32 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:32.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:32 smithi155 ceph-mon[33846]: from='client.36218 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-81636-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:32.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:32 smithi155 ceph-mon[33846]: osdmap e251: 8 total, 8 up, 8 in 2024-03-21T12:57:32.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:32 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3257114091' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-81636-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:32.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:32 smithi155 ceph-mon[33846]: from='client.36218 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-81636-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:32.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:32 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:32.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:32 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:32.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:32 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:32.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:32 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-81661-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-81661-49"}]': finished 2024-03-21T12:57:32.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:32 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:32.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:32 smithi043 ceph-mon[30062]: from='client.36218 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-81636-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:32.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:32 smithi043 ceph-mon[30062]: osdmap e251: 8 total, 8 up, 8 in 2024-03-21T12:57:32.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:32 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3257114091' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-81636-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:32.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:32 smithi043 ceph-mon[30062]: from='client.36218 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-81636-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:32.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:32 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:32.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:32 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:32.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:32 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:32.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:32 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi043-81661-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi043-81661-49"}]': finished 2024-03-21T12:57:32.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:32 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:32.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:32 smithi043 ceph-mon[41699]: from='client.36218 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi043-81636-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:32.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:32 smithi043 ceph-mon[41699]: osdmap e251: 8 total, 8 up, 8 in 2024-03-21T12:57:32.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:32 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3257114091' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-81636-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:32.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:32 smithi043 ceph-mon[41699]: from='client.36218 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-81636-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:32.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:32 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:32.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:32 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:32.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:32 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:33.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:33 smithi155 ceph-mon[33846]: pgmap v340: 364 pgs: 40 unknown, 324 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:57:33.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:33 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-33", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:57:33.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:33 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-33"}]: dispatch 2024-03-21T12:57:33.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:33 smithi155 ceph-mon[33846]: osdmap e252: 8 total, 8 up, 8 in 2024-03-21T12:57:33.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:33 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-33"}]: dispatch 2024-03-21T12:57:33.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:33 smithi043 ceph-mon[30062]: pgmap v340: 364 pgs: 40 unknown, 324 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:57:33.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:33 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-33", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:57:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:33 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-33"}]: dispatch 2024-03-21T12:57:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:33 smithi043 ceph-mon[30062]: osdmap e252: 8 total, 8 up, 8 in 2024-03-21T12:57:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:33 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-33"}]: dispatch 2024-03-21T12:57:33.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:33 smithi043 ceph-mon[41699]: pgmap v340: 364 pgs: 40 unknown, 324 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:57:33.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:33 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-33", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:57:33.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:33 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-33"}]: dispatch 2024-03-21T12:57:33.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:33 smithi043 ceph-mon[41699]: osdmap e252: 8 total, 8 up, 8 in 2024-03-21T12:57:33.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:33 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-33"}]: dispatch 2024-03-21T12:57:34.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:34 smithi155 ceph-mon[33846]: from='client.36218 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-81636-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-81636-40"}]': finished 2024-03-21T12:57:34.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:34 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-33"}]': finished 2024-03-21T12:57:34.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:34 smithi155 ceph-mon[33846]: osdmap e253: 8 total, 8 up, 8 in 2024-03-21T12:57:34.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:34 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-33", "mode": "writeback"}]: dispatch 2024-03-21T12:57:34.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:34 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-33", "mode": "writeback"}]: dispatch 2024-03-21T12:57:34.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:34 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-81661-49"}]: dispatch 2024-03-21T12:57:34.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:34 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:57:34.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:34 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:57:34.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:34 smithi043 ceph-mon[30062]: from='client.36218 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-81636-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-81636-40"}]': finished 2024-03-21T12:57:34.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:34 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-33"}]': finished 2024-03-21T12:57:34.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:34 smithi043 ceph-mon[30062]: osdmap e253: 8 total, 8 up, 8 in 2024-03-21T12:57:34.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:34 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-33", "mode": "writeback"}]: dispatch 2024-03-21T12:57:34.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:34 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-33", "mode": "writeback"}]: dispatch 2024-03-21T12:57:34.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:34 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-81661-49"}]: dispatch 2024-03-21T12:57:34.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:34 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:57:34.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:34 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:57:34.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:34 smithi043 ceph-mon[41699]: from='client.36218 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi043-81636-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi043-81636-40"}]': finished 2024-03-21T12:57:34.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:34 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-33"}]': finished 2024-03-21T12:57:34.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:34 smithi043 ceph-mon[41699]: osdmap e253: 8 total, 8 up, 8 in 2024-03-21T12:57:34.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:34 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-33", "mode": "writeback"}]: dispatch 2024-03-21T12:57:34.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:34 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-33", "mode": "writeback"}]: dispatch 2024-03-21T12:57:34.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:34 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-81661-49"}]: dispatch 2024-03-21T12:57:34.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:34 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:57:34.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:34 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:57:35.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:35 smithi155 ceph-mon[33846]: pgmap v343: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:57:35.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:35 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:57:35.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:35 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-33", "mode": "writeback"}]': finished 2024-03-21T12:57:35.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:35 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-81661-49"}]': finished 2024-03-21T12:57:35.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:35 smithi155 ceph-mon[33846]: osdmap e254: 8 total, 8 up, 8 in 2024-03-21T12:57:35.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:35 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-81661-49"}]: dispatch 2024-03-21T12:57:35.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:35 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:35.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:35 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:35 smithi043 ceph-mon[30062]: pgmap v343: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:57:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:35 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:57:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:35 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-33", "mode": "writeback"}]': finished 2024-03-21T12:57:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:35 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-81661-49"}]': finished 2024-03-21T12:57:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:35 smithi043 ceph-mon[30062]: osdmap e254: 8 total, 8 up, 8 in 2024-03-21T12:57:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:35 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-81661-49"}]: dispatch 2024-03-21T12:57:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:35 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:35 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:35.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:35 smithi043 ceph-mon[41699]: pgmap v343: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:57:35.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:35 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:57:35.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:35 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-33", "mode": "writeback"}]': finished 2024-03-21T12:57:35.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:35 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi043-81661-49"}]': finished 2024-03-21T12:57:35.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:35 smithi043 ceph-mon[41699]: osdmap e254: 8 total, 8 up, 8 in 2024-03-21T12:57:35.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:35 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-81661-49"}]: dispatch 2024-03-21T12:57:35.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:35 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:35.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:35 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:36.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-81661-49"}]': finished 2024-03-21T12:57:36.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:36 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:57:36.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:36 smithi155 ceph-mon[33846]: osdmap e255: 8 total, 8 up, 8 in 2024-03-21T12:57:36.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-33"}]: dispatch 2024-03-21T12:57:36.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3257114091' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:36.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:36 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-33"}]: dispatch 2024-03-21T12:57:36.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:36 smithi155 ceph-mon[33846]: from='client.36218 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:36.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2107061476' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:36.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:36 smithi155 ceph-mon[33846]: from='client.36224 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:36.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2107061476' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:36.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:36 smithi155 ceph-mon[33846]: from='client.36224 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:36.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2107061476' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-81661-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:36.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:36 smithi155 ceph-mon[33846]: from='client.36224 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-81661-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-81661-49"}]': finished 2024-03-21T12:57:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:57:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[30062]: osdmap e255: 8 total, 8 up, 8 in 2024-03-21T12:57:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-33"}]: dispatch 2024-03-21T12:57:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3257114091' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-33"}]: dispatch 2024-03-21T12:57:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[30062]: from='client.36218 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2107061476' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[30062]: from='client.36224 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2107061476' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[30062]: from='client.36224 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:36.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2107061476' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-81661-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:36.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[30062]: from='client.36224 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-81661-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:36.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2943115900' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi043-81661-49"}]': finished 2024-03-21T12:57:36.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:57:36.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[41699]: osdmap e255: 8 total, 8 up, 8 in 2024-03-21T12:57:36.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-33"}]: dispatch 2024-03-21T12:57:36.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3257114091' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:36.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-33"}]: dispatch 2024-03-21T12:57:36.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[41699]: from='client.36218 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:36.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2107061476' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:36.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[41699]: from='client.36224 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:36.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2107061476' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:36.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[41699]: from='client.36224 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:36.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2107061476' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-81661-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:36.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:36 smithi043 ceph-mon[41699]: from='client.36224 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-81661-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:37.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:37 smithi155 ceph-mon[33846]: pgmap v346: 356 pgs: 356 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:57:37.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:37 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:57:37.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:37 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-33"}]': finished 2024-03-21T12:57:37.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:37 smithi155 ceph-mon[33846]: from='client.36218 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81636-40"}]': finished 2024-03-21T12:57:37.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:37 smithi155 ceph-mon[33846]: from='client.36224 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-81661-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:37.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:37 smithi155 ceph-mon[33846]: osdmap e256: 8 total, 8 up, 8 in 2024-03-21T12:57:37.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:37 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3257114091' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:37.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:37 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2107061476' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-81661-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:37.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:37 smithi155 ceph-mon[33846]: from='client.36218 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:37.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:37 smithi155 ceph-mon[33846]: from='client.36224 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-81661-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:37.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:37 smithi043 ceph-mon[30062]: pgmap v346: 356 pgs: 356 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:57:37.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:37 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:57:37.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:37 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-33"}]': finished 2024-03-21T12:57:37.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:37 smithi043 ceph-mon[30062]: from='client.36218 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81636-40"}]': finished 2024-03-21T12:57:37.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:37 smithi043 ceph-mon[30062]: from='client.36224 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-81661-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:37.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:37 smithi043 ceph-mon[30062]: osdmap e256: 8 total, 8 up, 8 in 2024-03-21T12:57:37.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:37 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3257114091' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:37.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:37 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2107061476' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-81661-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:37.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:37 smithi043 ceph-mon[30062]: from='client.36218 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:37.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:37 smithi043 ceph-mon[30062]: from='client.36224 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-81661-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:37.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:37 smithi043 ceph-mon[41699]: pgmap v346: 356 pgs: 356 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:57:37.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:37 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:57:37.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:37 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-33"}]': finished 2024-03-21T12:57:37.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:37 smithi043 ceph-mon[41699]: from='client.36218 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi043-81636-40"}]': finished 2024-03-21T12:57:37.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:37 smithi043 ceph-mon[41699]: from='client.36224 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi043-81661-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:37.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:37 smithi043 ceph-mon[41699]: osdmap e256: 8 total, 8 up, 8 in 2024-03-21T12:57:37.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:37 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3257114091' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:37.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:37 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2107061476' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-81661-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:37.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:37 smithi043 ceph-mon[41699]: from='client.36218 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81636-40"}]: dispatch 2024-03-21T12:57:37.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:37 smithi043 ceph-mon[41699]: from='client.36224 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-81661-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:38.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:57:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:57:38.645 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:57:38 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:57:38.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:38 smithi155 ceph-mon[33846]: from='client.36218 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81636-40"}]': finished 2024-03-21T12:57:38.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:38 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2486641975' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:38.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:38 smithi155 ceph-mon[33846]: osdmap e257: 8 total, 8 up, 8 in 2024-03-21T12:57:38.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:38 smithi155 ceph-mon[33846]: from='client.36230 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:38.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:38 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2486641975' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:38.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:38 smithi155 ceph-mon[33846]: from='client.36230 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:38.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:38 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2486641975' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-81636-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:38.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:38 smithi155 ceph-mon[33846]: from='client.36230 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-81636-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:38.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:38 smithi155 ceph-mon[33846]: from='client.36224 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-81661-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-81661-50"}]': finished 2024-03-21T12:57:38.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:38 smithi155 ceph-mon[33846]: from='client.36230 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-81636-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:38.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:38 smithi155 ceph-mon[33846]: osdmap e258: 8 total, 8 up, 8 in 2024-03-21T12:57:38.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:38 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2486641975' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-81636-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:38.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:38 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:38.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:38 smithi155 ceph-mon[33846]: from='client.36230 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-81636-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:38.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:38 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:38.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[41699]: from='client.36218 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81636-40"}]': finished 2024-03-21T12:57:38.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2486641975' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:38.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[41699]: osdmap e257: 8 total, 8 up, 8 in 2024-03-21T12:57:38.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[41699]: from='client.36230 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:38.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2486641975' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:38.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[41699]: from='client.36230 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:38.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2486641975' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-81636-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:38.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[41699]: from='client.36230 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-81636-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:38.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[41699]: from='client.36224 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-81661-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-81661-50"}]': finished 2024-03-21T12:57:38.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[41699]: from='client.36230 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-81636-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:38.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[41699]: osdmap e258: 8 total, 8 up, 8 in 2024-03-21T12:57:38.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2486641975' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-81636-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:38.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:38.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[41699]: from='client.36230 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-81636-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:38.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:38.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[30062]: from='client.36218 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi043-81636-40"}]': finished 2024-03-21T12:57:38.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2486641975' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:38.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[30062]: osdmap e257: 8 total, 8 up, 8 in 2024-03-21T12:57:38.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[30062]: from='client.36230 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:38.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2486641975' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:38.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[30062]: from='client.36230 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:38.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2486641975' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-81636-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:38.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[30062]: from='client.36230 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-81636-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:38.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[30062]: from='client.36224 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi043-81661-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi043-81661-50"}]': finished 2024-03-21T12:57:38.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[30062]: from='client.36230 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi043-81636-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:38.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[30062]: osdmap e258: 8 total, 8 up, 8 in 2024-03-21T12:57:38.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2486641975' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-81636-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:38.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:38.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[30062]: from='client.36230 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-81636-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:38.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:38 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:39.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:39 smithi155 ceph-mon[33846]: pgmap v349: 324 pgs: 324 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:57:39.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:39 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:57:39.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:39 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:39.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:39 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:39.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:39 smithi155 ceph-mon[33846]: osdmap e259: 8 total, 8 up, 8 in 2024-03-21T12:57:39.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:39 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:39 smithi043 ceph-mon[30062]: pgmap v349: 324 pgs: 324 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:57:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:39 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:57:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:39 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:39 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:39 smithi043 ceph-mon[30062]: osdmap e259: 8 total, 8 up, 8 in 2024-03-21T12:57:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:39 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:39.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:39 smithi043 ceph-mon[41699]: pgmap v349: 324 pgs: 324 active+clean; 4.6 MiB data, 598 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:57:39.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:39 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:57:39.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:39 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:39.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:39 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:39.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:39 smithi043 ceph-mon[41699]: osdmap e259: 8 total, 8 up, 8 in 2024-03-21T12:57:39.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:39 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:41.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:41 smithi155 ceph-mon[33846]: pgmap v352: 364 pgs: 40 unknown, 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:57:41.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:41 smithi155 ceph-mon[33846]: from='client.36230 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-81636-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-81636-41"}]': finished 2024-03-21T12:57:41.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:41 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-35", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:57:41.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:41 smithi155 ceph-mon[33846]: osdmap e260: 8 total, 8 up, 8 in 2024-03-21T12:57:41.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-35"}]: dispatch 2024-03-21T12:57:41.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:41 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-35"}]: dispatch 2024-03-21T12:57:41.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2107061476' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:41.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:41 smithi155 ceph-mon[33846]: from='client.36224 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:41.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:41 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:41 smithi043 ceph-mon[41699]: pgmap v352: 364 pgs: 40 unknown, 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:57:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:41 smithi043 ceph-mon[41699]: from='client.36230 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-81636-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-81636-41"}]': finished 2024-03-21T12:57:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:41 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-35", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:57:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:41 smithi043 ceph-mon[41699]: osdmap e260: 8 total, 8 up, 8 in 2024-03-21T12:57:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-35"}]: dispatch 2024-03-21T12:57:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:41 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-35"}]: dispatch 2024-03-21T12:57:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2107061476' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:41 smithi043 ceph-mon[41699]: from='client.36224 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:41 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:41.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:41 smithi043 ceph-mon[30062]: pgmap v352: 364 pgs: 40 unknown, 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:57:41.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:41 smithi043 ceph-mon[30062]: from='client.36230 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi043-81636-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi043-81636-41"}]': finished 2024-03-21T12:57:41.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:41 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-35", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:57:41.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:41 smithi043 ceph-mon[30062]: osdmap e260: 8 total, 8 up, 8 in 2024-03-21T12:57:41.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-35"}]: dispatch 2024-03-21T12:57:41.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:41 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-35"}]: dispatch 2024-03-21T12:57:41.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2107061476' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:41.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:41 smithi043 ceph-mon[30062]: from='client.36224 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:41.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:41 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:42.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:42 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-35"}]': finished 2024-03-21T12:57:42.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:42 smithi155 ceph-mon[33846]: from='client.36224 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81661-50"}]': finished 2024-03-21T12:57:42.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:42 smithi155 ceph-mon[33846]: osdmap e261: 8 total, 8 up, 8 in 2024-03-21T12:57:42.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-35", "mode": "writeback"}]: dispatch 2024-03-21T12:57:42.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2107061476' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:42.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:42 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-35", "mode": "writeback"}]: dispatch 2024-03-21T12:57:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:42 smithi155 ceph-mon[33846]: from='client.36224 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:42 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:57:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:42 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-35", "mode": "writeback"}]': finished 2024-03-21T12:57:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:42 smithi155 ceph-mon[33846]: from='client.36224 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81661-50"}]': finished 2024-03-21T12:57:42.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:42 smithi155 ceph-mon[33846]: osdmap e262: 8 total, 8 up, 8 in 2024-03-21T12:57:42.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:42 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-35"}]': finished 2024-03-21T12:57:42.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:42 smithi043 ceph-mon[30062]: from='client.36224 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81661-50"}]': finished 2024-03-21T12:57:42.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:42 smithi043 ceph-mon[30062]: osdmap e261: 8 total, 8 up, 8 in 2024-03-21T12:57:42.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-35", "mode": "writeback"}]: dispatch 2024-03-21T12:57:42.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2107061476' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:42.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:42 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-35", "mode": "writeback"}]: dispatch 2024-03-21T12:57:42.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:42 smithi043 ceph-mon[30062]: from='client.36224 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:42.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:42 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:57:42.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:42 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-35", "mode": "writeback"}]': finished 2024-03-21T12:57:42.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:42 smithi043 ceph-mon[30062]: from='client.36224 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81661-50"}]': finished 2024-03-21T12:57:42.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:42 smithi043 ceph-mon[30062]: osdmap e262: 8 total, 8 up, 8 in 2024-03-21T12:57:42.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:42 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-35"}]': finished 2024-03-21T12:57:42.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:42 smithi043 ceph-mon[41699]: from='client.36224 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi043-81661-50"}]': finished 2024-03-21T12:57:42.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:42 smithi043 ceph-mon[41699]: osdmap e261: 8 total, 8 up, 8 in 2024-03-21T12:57:42.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-35", "mode": "writeback"}]: dispatch 2024-03-21T12:57:42.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2107061476' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:42.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:42 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-35", "mode": "writeback"}]: dispatch 2024-03-21T12:57:42.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:42 smithi043 ceph-mon[41699]: from='client.36224 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81661-50"}]: dispatch 2024-03-21T12:57:42.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:42 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:57:42.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:42 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-35", "mode": "writeback"}]': finished 2024-03-21T12:57:42.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:42 smithi043 ceph-mon[41699]: from='client.36224 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi043-81661-50"}]': finished 2024-03-21T12:57:42.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:42 smithi043 ceph-mon[41699]: osdmap e262: 8 total, 8 up, 8 in 2024-03-21T12:57:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:43 smithi155 ceph-mon[33846]: pgmap v355: 364 pgs: 40 unknown, 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:57:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2486641975' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:43 smithi155 ceph-mon[33846]: from='client.36230 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:43 smithi155 ceph-mon[33846]: from='client.36230 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-81636-41"}]': finished 2024-03-21T12:57:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:43 smithi155 ceph-mon[33846]: osdmap e263: 8 total, 8 up, 8 in 2024-03-21T12:57:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2486641975' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:43 smithi155 ceph-mon[33846]: from='client.36230 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:43.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3589191459' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81661-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:43 smithi043 ceph-mon[30062]: pgmap v355: 364 pgs: 40 unknown, 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:57:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2486641975' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:43 smithi043 ceph-mon[30062]: from='client.36230 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:43 smithi043 ceph-mon[30062]: from='client.36230 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-81636-41"}]': finished 2024-03-21T12:57:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:43 smithi043 ceph-mon[30062]: osdmap e263: 8 total, 8 up, 8 in 2024-03-21T12:57:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2486641975' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:43 smithi043 ceph-mon[30062]: from='client.36230 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3589191459' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81661-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:43.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:43 smithi043 ceph-mon[41699]: pgmap v355: 364 pgs: 40 unknown, 324 active+clean; 4.6 MiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:57:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2486641975' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:43 smithi043 ceph-mon[41699]: from='client.36230 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:43 smithi043 ceph-mon[41699]: from='client.36230 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi043-81636-41"}]': finished 2024-03-21T12:57:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:43 smithi043 ceph-mon[41699]: osdmap e263: 8 total, 8 up, 8 in 2024-03-21T12:57:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2486641975' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:43 smithi043 ceph-mon[41699]: from='client.36230 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-81636-41"}]: dispatch 2024-03-21T12:57:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3589191459' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81661-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:45.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:45 smithi155 ceph-mon[33846]: pgmap v358: 388 pgs: 32 unknown, 356 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:57:45.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:45 smithi155 ceph-mon[33846]: from='client.36230 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-81636-41"}]': finished 2024-03-21T12:57:45.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:45 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3589191459' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81661-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:45.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:45 smithi155 ceph-mon[33846]: osdmap e264: 8 total, 8 up, 8 in 2024-03-21T12:57:45.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:45 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/538777427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:45.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:45 smithi155 ceph-mon[33846]: from='client.26236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:45.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:45 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/538777427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:45.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:45 smithi155 ceph-mon[33846]: from='client.26236 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:45.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:45 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/538777427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-81636-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:45.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:45 smithi155 ceph-mon[33846]: from='client.26236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-81636-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:45.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:45 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:45.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:45 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:45 smithi043 ceph-mon[30062]: pgmap v358: 388 pgs: 32 unknown, 356 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:57:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:45 smithi043 ceph-mon[30062]: from='client.36230 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-81636-41"}]': finished 2024-03-21T12:57:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3589191459' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81661-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:45 smithi043 ceph-mon[30062]: osdmap e264: 8 total, 8 up, 8 in 2024-03-21T12:57:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/538777427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:45 smithi043 ceph-mon[30062]: from='client.26236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/538777427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:45 smithi043 ceph-mon[30062]: from='client.26236 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/538777427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-81636-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:45 smithi043 ceph-mon[30062]: from='client.26236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-81636-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:45.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:45.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:45 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:45.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:45 smithi043 ceph-mon[41699]: pgmap v358: 388 pgs: 32 unknown, 356 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:57:45.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:45 smithi043 ceph-mon[41699]: from='client.36230 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi043-81636-41"}]': finished 2024-03-21T12:57:45.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:45 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3589191459' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi043-81661-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:45.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:45 smithi043 ceph-mon[41699]: osdmap e264: 8 total, 8 up, 8 in 2024-03-21T12:57:45.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:45 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/538777427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:45.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:45 smithi043 ceph-mon[41699]: from='client.26236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:45.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:45 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/538777427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:45.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:45 smithi043 ceph-mon[41699]: from='client.26236 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:45.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:45 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/538777427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-81636-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:45.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:45 smithi043 ceph-mon[41699]: from='client.26236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-81636-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:45.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:45 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:45.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:45 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:46.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:46 smithi155 ceph-mon[33846]: from='client.26236 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-81636-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:46.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:46 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:57:46.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:46 smithi155 ceph-mon[33846]: osdmap e265: 8 total, 8 up, 8 in 2024-03-21T12:57:46.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/538777427' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi043-81636-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:46 smithi155 ceph-mon[33846]: from='client.26236 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi043-81636-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-35"}]: dispatch 2024-03-21T12:57:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:46 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-35"}]: dispatch 2024-03-21T12:57:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/284996261' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:46 smithi155 ceph-mon[33846]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/284996261' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:46 smithi155 ceph-mon[33846]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/284996261' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-81661-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:46 smithi155 ceph-mon[33846]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-81661-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:46 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-35"}]': finished 2024-03-21T12:57:46.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[30062]: from='client.26236 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-81636-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:46.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:57:46.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[30062]: osdmap e265: 8 total, 8 up, 8 in 2024-03-21T12:57:46.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/538777427' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi043-81636-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:46.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[30062]: from='client.26236 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi043-81636-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:46.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-35"}]: dispatch 2024-03-21T12:57:46.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-35"}]: dispatch 2024-03-21T12:57:46.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/284996261' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:46.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[30062]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:46.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/284996261' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:46.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[30062]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:46.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/284996261' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-81661-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:46.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[30062]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-81661-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:46.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-35"}]': finished 2024-03-21T12:57:46.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[41699]: from='client.26236 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi043-81636-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:46.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:57:46.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[41699]: osdmap e265: 8 total, 8 up, 8 in 2024-03-21T12:57:46.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/538777427' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi043-81636-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:46.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[41699]: from='client.26236 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi043-81636-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:46.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-35"}]: dispatch 2024-03-21T12:57:46.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-35"}]: dispatch 2024-03-21T12:57:46.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/284996261' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:46.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[41699]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:46.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/284996261' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:46.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[41699]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:46.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/284996261' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-81661-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:46.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[41699]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-81661-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:46.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:46 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-35"}]': finished 2024-03-21T12:57:47.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:47 smithi155 ceph-mon[33846]: pgmap v361: 356 pgs: 356 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:57:47.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:47 smithi155 ceph-mon[33846]: from='client.36248 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-81661-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:47.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/284996261' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-81661-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:47.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:47 smithi155 ceph-mon[33846]: osdmap e266: 8 total, 8 up, 8 in 2024-03-21T12:57:47.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:47 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:47.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:47 smithi155 ceph-mon[33846]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-81661-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:47.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:47 smithi155 ceph-mon[33846]: from='client.26236 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi043-81636-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-81636-42"}]': finished 2024-03-21T12:57:47.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:47.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:47 smithi155 ceph-mon[33846]: osdmap e267: 8 total, 8 up, 8 in 2024-03-21T12:57:47.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:47 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:47.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:47 smithi043 ceph-mon[41699]: pgmap v361: 356 pgs: 356 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:57:47.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:47 smithi043 ceph-mon[41699]: from='client.36248 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-81661-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:47.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/284996261' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-81661-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:47.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:47 smithi043 ceph-mon[41699]: osdmap e266: 8 total, 8 up, 8 in 2024-03-21T12:57:47.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:47 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:47.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:47 smithi043 ceph-mon[41699]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-81661-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:47.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:47 smithi043 ceph-mon[41699]: from='client.26236 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi043-81636-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-81636-42"}]': finished 2024-03-21T12:57:47.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:47.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:47 smithi043 ceph-mon[41699]: osdmap e267: 8 total, 8 up, 8 in 2024-03-21T12:57:47.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:47 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:47.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:47 smithi043 ceph-mon[30062]: pgmap v361: 356 pgs: 356 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:57:47.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:47 smithi043 ceph-mon[30062]: from='client.36248 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi043-81661-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:47.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/284996261' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-81661-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:47.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:47 smithi043 ceph-mon[30062]: osdmap e266: 8 total, 8 up, 8 in 2024-03-21T12:57:47.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:47 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:47.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:47 smithi043 ceph-mon[30062]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-81661-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:47.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:47 smithi043 ceph-mon[30062]: from='client.26236 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi043-81636-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi043-81636-42"}]': finished 2024-03-21T12:57:47.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:47.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:47 smithi043 ceph-mon[30062]: osdmap e267: 8 total, 8 up, 8 in 2024-03-21T12:57:47.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:47 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:48.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:57:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:57:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:57:48.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:57:48 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:57:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:49 smithi155 ceph-mon[33846]: pgmap v364: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:57:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:49 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:57:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:49 smithi155 ceph-mon[33846]: from='client.36248 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-81661-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-81661-52"}]': finished 2024-03-21T12:57:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:49 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:57:49.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:49 smithi155 ceph-mon[33846]: osdmap e268: 8 total, 8 up, 8 in 2024-03-21T12:57:49.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:49 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-35"}]: dispatch 2024-03-21T12:57:49.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:49 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-35"}]: dispatch 2024-03-21T12:57:49.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:49 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:57:49.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:49 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:57:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:49 smithi043 ceph-mon[30062]: pgmap v364: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:57:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:49 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:57:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:49 smithi043 ceph-mon[30062]: from='client.36248 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-81661-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-81661-52"}]': finished 2024-03-21T12:57:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:49 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:57:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:49 smithi043 ceph-mon[30062]: osdmap e268: 8 total, 8 up, 8 in 2024-03-21T12:57:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:49 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-35"}]: dispatch 2024-03-21T12:57:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:49 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-35"}]: dispatch 2024-03-21T12:57:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:49 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:57:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:49 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:57:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:49 smithi043 ceph-mon[41699]: pgmap v364: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 600 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:57:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:49 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:57:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:49 smithi043 ceph-mon[41699]: from='client.36248 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi043-81661-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi043-81661-52"}]': finished 2024-03-21T12:57:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:49 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:57:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:49 smithi043 ceph-mon[41699]: osdmap e268: 8 total, 8 up, 8 in 2024-03-21T12:57:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:49 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-35"}]: dispatch 2024-03-21T12:57:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:49 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-35"}]: dispatch 2024-03-21T12:57:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:49 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:57:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:49 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:57:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:50 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:57:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:50 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-35"}]': finished 2024-03-21T12:57:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:50 smithi155 ceph-mon[33846]: osdmap e269: 8 total, 8 up, 8 in 2024-03-21T12:57:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:50 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/538777427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:50 smithi155 ceph-mon[33846]: from='client.26236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:50 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:57:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:50 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-35"}]': finished 2024-03-21T12:57:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:50 smithi043 ceph-mon[30062]: osdmap e269: 8 total, 8 up, 8 in 2024-03-21T12:57:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:50 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/538777427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:50 smithi043 ceph-mon[30062]: from='client.26236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:50 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:57:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:50 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-35"}]': finished 2024-03-21T12:57:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:50 smithi043 ceph-mon[41699]: osdmap e269: 8 total, 8 up, 8 in 2024-03-21T12:57:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:50 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/538777427' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:50.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:50 smithi043 ceph-mon[41699]: from='client.26236 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:51.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: sta api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7018 ms) 2024-03-21T12:57:51.433 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplete 2024-03-21T12:57:51.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (6986 ms) 2024-03-21T12:57:51.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-03-21T12:57:51.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (7017 ms) 2024-03-21T12:57:51.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-03-21T12:57:51.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (7009 ms) 2024-03-21T12:57:51.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-03-21T12:57:51.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7018 ms) 2024-03-21T12:57:51.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-03-21T12:57:51.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7045 ms) 2024-03-21T12:57:51.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-03-21T12:57:51.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7015 ms) 2024-03-21T12:57:51.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-03-21T12:57:51.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (6910 ms) 2024-03-21T12:57:51.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-03-21T12:57:51.434 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (7014 ms) 2024-03-21T12:57:51.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-03-21T12:57:51.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7028 ms) 2024-03-21T12:57:51.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-03-21T12:57:51.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (6967 ms) 2024-03-21T12:57:51.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-03-21T12:57:51.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (7028 ms) 2024-03-21T12:57:51.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-03-21T12:57:51.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (6973 ms) 2024-03-21T12:57:51.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-03-21T12:57:51.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7037 ms) 2024-03-21T12:57:51.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-03-21T12:57:51.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7015 ms) 2024-03-21T12:57:51.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-03-21T12:57:51.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (7026 ms) 2024-03-21T12:57:51.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (112109 ms total) 2024-03-21T12:57:51.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: 2024-03-21T12:57:51.435 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [----------] Global test environment tear-down 2024-03-21T12:57:51.436 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [==========] 42 tests from 2 test suites ran. (205828 ms total) 2024-03-21T12:57:51.436 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio: [ PASSED ] 42 tests. 2024-03-21T12:57:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:51 smithi155 ceph-mon[33846]: pgmap v367: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-21T12:57:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:51 smithi155 ceph-mon[33846]: from='client.26236 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-81636-42"}]': finished 2024-03-21T12:57:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:51 smithi155 ceph-mon[33846]: osdmap e270: 8 total, 8 up, 8 in 2024-03-21T12:57:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:51 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/284996261' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:51 smithi155 ceph-mon[33846]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:51 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/538777427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:51 smithi155 ceph-mon[33846]: from='client.26236 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:51.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:51 smithi043 ceph-mon[30062]: pgmap v367: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-21T12:57:51.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:51 smithi043 ceph-mon[30062]: from='client.26236 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-81636-42"}]': finished 2024-03-21T12:57:51.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:51 smithi043 ceph-mon[30062]: osdmap e270: 8 total, 8 up, 8 in 2024-03-21T12:57:51.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:51 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/284996261' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:51.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:51 smithi043 ceph-mon[30062]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:51.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:51 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/538777427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:51.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:51 smithi043 ceph-mon[30062]: from='client.26236 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:51.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:51 smithi043 ceph-mon[41699]: pgmap v367: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-21T12:57:51.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:51 smithi043 ceph-mon[41699]: from='client.26236 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi043-81636-42"}]': finished 2024-03-21T12:57:51.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:51 smithi043 ceph-mon[41699]: osdmap e270: 8 total, 8 up, 8 in 2024-03-21T12:57:51.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:51 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/284996261' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:51.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:51 smithi043 ceph-mon[41699]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:51.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:51 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/538777427' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:51.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:51 smithi043 ceph-mon[41699]: from='client.26236 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-81636-42"}]: dispatch 2024-03-21T12:57:52.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:52 smithi155 ceph-mon[33846]: from='client.36248 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-81661-52"}]': finished 2024-03-21T12:57:52.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:52 smithi155 ceph-mon[33846]: from='client.26236 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-81636-42"}]': finished 2024-03-21T12:57:52.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:52 smithi155 ceph-mon[33846]: osdmap e271: 8 total, 8 up, 8 in 2024-03-21T12:57:52.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/284996261' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:52.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:52.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:52 smithi155 ceph-mon[33846]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:52.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:52 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:52.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:52 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:52.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:52 smithi155 ceph-mon[33846]: from='client.36248 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-81661-52"}]': finished 2024-03-21T12:57:52.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:52 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:52.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:52 smithi155 ceph-mon[33846]: osdmap e272: 8 total, 8 up, 8 in 2024-03-21T12:57:52.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:52.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:52 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:52.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/954503638' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:52.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:52 smithi155 ceph-mon[33846]: from='client.36254 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:52.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/954503638' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:52.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:52 smithi155 ceph-mon[33846]: from='client.36254 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:52.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/954503638' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-81661-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:52.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:52 smithi155 ceph-mon[33846]: from='client.36254 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-81661-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:52.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[30062]: from='client.36248 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-81661-52"}]': finished 2024-03-21T12:57:52.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[30062]: from='client.26236 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-81636-42"}]': finished 2024-03-21T12:57:52.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[30062]: osdmap e271: 8 total, 8 up, 8 in 2024-03-21T12:57:52.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/284996261' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:52.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:52.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[30062]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:52.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:52.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:52.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[30062]: from='client.36248 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-81661-52"}]': finished 2024-03-21T12:57:52.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:52.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[30062]: osdmap e272: 8 total, 8 up, 8 in 2024-03-21T12:57:52.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:52.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:52.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/954503638' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:52.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[30062]: from='client.36254 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:52.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/954503638' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:52.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[30062]: from='client.36254 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:52.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/954503638' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-81661-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:52.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[30062]: from='client.36254 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-81661-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:52.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[41699]: from='client.36248 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi043-81661-52"}]': finished 2024-03-21T12:57:52.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[41699]: from='client.26236 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi043-81636-42"}]': finished 2024-03-21T12:57:52.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[41699]: osdmap e271: 8 total, 8 up, 8 in 2024-03-21T12:57:52.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/284996261' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:52.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:52.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[41699]: from='client.36248 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-81661-52"}]: dispatch 2024-03-21T12:57:52.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:57:52.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:52.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[41699]: from='client.36248 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi043-81661-52"}]': finished 2024-03-21T12:57:52.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:57:52.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[41699]: osdmap e272: 8 total, 8 up, 8 in 2024-03-21T12:57:52.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:52.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:57:52.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/954503638' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:52.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[41699]: from='client.36254 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:52.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/954503638' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:52.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[41699]: from='client.36254 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:52.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/954503638' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-81661-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:52.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:52 smithi043 ceph-mon[41699]: from='client.36254 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-81661-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:57:53.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:53 smithi155 ceph-mon[33846]: pgmap v370: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T12:57:53.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:53 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-37", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:57:53.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:53 smithi155 ceph-mon[33846]: from='client.36254 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-81661-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:53.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:53 smithi155 ceph-mon[33846]: osdmap e273: 8 total, 8 up, 8 in 2024-03-21T12:57:53.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:53 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-37"}]: dispatch 2024-03-21T12:57:53.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:53 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/954503638' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-81661-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:53.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:53 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-37"}]: dispatch 2024-03-21T12:57:53.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:53 smithi155 ceph-mon[33846]: from='client.36254 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-81661-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:53.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:53 smithi043 ceph-mon[30062]: pgmap v370: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T12:57:53.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:53 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-37", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:57:53.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:53 smithi043 ceph-mon[30062]: from='client.36254 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-81661-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:53.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:53 smithi043 ceph-mon[30062]: osdmap e273: 8 total, 8 up, 8 in 2024-03-21T12:57:53.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:53 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-37"}]: dispatch 2024-03-21T12:57:53.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:53 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/954503638' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-81661-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:53.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:53 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-37"}]: dispatch 2024-03-21T12:57:53.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:53 smithi043 ceph-mon[30062]: from='client.36254 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-81661-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:53.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:53 smithi043 ceph-mon[41699]: pgmap v370: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T12:57:53.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:53 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-37", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:57:53.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:53 smithi043 ceph-mon[41699]: from='client.36254 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi043-81661-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:57:53.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:53 smithi043 ceph-mon[41699]: osdmap e273: 8 total, 8 up, 8 in 2024-03-21T12:57:53.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:53 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-37"}]: dispatch 2024-03-21T12:57:53.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:53 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/954503638' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-81661-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:53.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:53 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-37"}]: dispatch 2024-03-21T12:57:53.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:53 smithi043 ceph-mon[41699]: from='client.36254 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-81661-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:55.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:55 smithi155 ceph-mon[33846]: pgmap v373: 356 pgs: 356 active+clean; 4.6 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-21T12:57:55.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:55 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-37"}]': finished 2024-03-21T12:57:55.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:55 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-37", "mode": "writeback"}]: dispatch 2024-03-21T12:57:55.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:55 smithi155 ceph-mon[33846]: osdmap e274: 8 total, 8 up, 8 in 2024-03-21T12:57:55.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:55 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-37", "mode": "writeback"}]: dispatch 2024-03-21T12:57:55.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:55 smithi043 ceph-mon[30062]: pgmap v373: 356 pgs: 356 active+clean; 4.6 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-21T12:57:55.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:55 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-37"}]': finished 2024-03-21T12:57:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:55 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-37", "mode": "writeback"}]: dispatch 2024-03-21T12:57:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:55 smithi043 ceph-mon[30062]: osdmap e274: 8 total, 8 up, 8 in 2024-03-21T12:57:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:55 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-37", "mode": "writeback"}]: dispatch 2024-03-21T12:57:55.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:55 smithi043 ceph-mon[41699]: pgmap v373: 356 pgs: 356 active+clean; 4.6 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-21T12:57:55.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:55 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-37"}]': finished 2024-03-21T12:57:55.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:55 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-37", "mode": "writeback"}]: dispatch 2024-03-21T12:57:55.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:55 smithi043 ceph-mon[41699]: osdmap e274: 8 total, 8 up, 8 in 2024-03-21T12:57:55.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:55 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-37", "mode": "writeback"}]: dispatch 2024-03-21T12:57:56.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:56 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:57:56.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:56 smithi155 ceph-mon[33846]: from='client.36254 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-81661-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-81661-53"}]': finished 2024-03-21T12:57:56.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:56 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-37", "mode": "writeback"}]': finished 2024-03-21T12:57:56.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:56 smithi155 ceph-mon[33846]: osdmap e275: 8 total, 8 up, 8 in 2024-03-21T12:57:56.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:56 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:57:56.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:56 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:56.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:56 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:56.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:56 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:57:56.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:56 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:57:56.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:56 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:57:56.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:56 smithi043 ceph-mon[30062]: from='client.36254 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-81661-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-81661-53"}]': finished 2024-03-21T12:57:56.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:56 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-37", "mode": "writeback"}]': finished 2024-03-21T12:57:56.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:56 smithi043 ceph-mon[30062]: osdmap e275: 8 total, 8 up, 8 in 2024-03-21T12:57:56.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:56 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:57:56.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:56 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:56.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:56 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:56.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:56 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:57:56.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:56 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:57:56.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:56 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:57:56.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:56 smithi043 ceph-mon[41699]: from='client.36254 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi043-81661-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi043-81661-53"}]': finished 2024-03-21T12:57:56.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:56 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-37", "mode": "writeback"}]': finished 2024-03-21T12:57:56.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:56 smithi043 ceph-mon[41699]: osdmap e275: 8 total, 8 up, 8 in 2024-03-21T12:57:56.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:56 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:57:56.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:56 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:56.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:56 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:57:56.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:56 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:57:56.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:56 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:57:57.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:57 smithi155 ceph-mon[33846]: pgmap v376: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T12:57:57.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:57 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:57:57.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:57 smithi155 ceph-mon[33846]: osdmap e276: 8 total, 8 up, 8 in 2024-03-21T12:57:57.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:57 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-37"}]: dispatch 2024-03-21T12:57:57.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:57 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-37"}]: dispatch 2024-03-21T12:57:57.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:57 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:57:57.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:57 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:57:57.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:57 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:57:57.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:57 smithi043 ceph-mon[30062]: pgmap v376: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T12:57:57.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:57 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:57:57.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:57 smithi043 ceph-mon[30062]: osdmap e276: 8 total, 8 up, 8 in 2024-03-21T12:57:57.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:57 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-37"}]: dispatch 2024-03-21T12:57:57.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:57 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-37"}]: dispatch 2024-03-21T12:57:57.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:57 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:57:57.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:57 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:57:57.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:57 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:57:57.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:57 smithi043 ceph-mon[41699]: pgmap v376: 364 pgs: 8 unknown, 356 active+clean; 4.6 MiB data, 601 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T12:57:57.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:57 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:57:57.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:57 smithi043 ceph-mon[41699]: osdmap e276: 8 total, 8 up, 8 in 2024-03-21T12:57:57.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:57 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-37"}]: dispatch 2024-03-21T12:57:57.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:57 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-37"}]: dispatch 2024-03-21T12:57:57.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:57 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:57:57.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:57 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:57:57.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:57 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:57:58.367 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:57:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:57:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:57:58.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:57:58 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:57:58.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:58 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:57:58.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:58 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-37"}]': finished 2024-03-21T12:57:58.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:58 smithi155 ceph-mon[33846]: osdmap e277: 8 total, 8 up, 8 in 2024-03-21T12:57:58.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:58 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/954503638' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:58.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:58 smithi155 ceph-mon[33846]: from='client.36254 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:58.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:58 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:57:58.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:58 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-37"}]': finished 2024-03-21T12:57:58.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:58 smithi043 ceph-mon[30062]: osdmap e277: 8 total, 8 up, 8 in 2024-03-21T12:57:58.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:58 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/954503638' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:58.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:58 smithi043 ceph-mon[30062]: from='client.36254 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:58.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:58 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:57:58.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:58 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-37"}]': finished 2024-03-21T12:57:58.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:58 smithi043 ceph-mon[41699]: osdmap e277: 8 total, 8 up, 8 in 2024-03-21T12:57:58.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:58 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/954503638' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:58.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:58 smithi043 ceph-mon[41699]: from='client.36254 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:59.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:59 smithi155 ceph-mon[33846]: pgmap v379: 356 pgs: 356 active+clean; 4.6 MiB data, 601 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:57:59.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:59 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:57:59.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:59 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:59.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:59 smithi155 ceph-mon[33846]: from='client.36254 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81661-53"}]': finished 2024-03-21T12:57:59.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:59 smithi155 ceph-mon[33846]: osdmap e278: 8 total, 8 up, 8 in 2024-03-21T12:57:59.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:59 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/954503638' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:59.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:57:59 smithi155 ceph-mon[33846]: from='client.36254 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:59.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:59 smithi043 ceph-mon[30062]: pgmap v379: 356 pgs: 356 active+clean; 4.6 MiB data, 601 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:57:59.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:59 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:57:59.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:59 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:59.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:59 smithi043 ceph-mon[30062]: from='client.36254 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81661-53"}]': finished 2024-03-21T12:57:59.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:59 smithi043 ceph-mon[30062]: osdmap e278: 8 total, 8 up, 8 in 2024-03-21T12:57:59.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:59 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/954503638' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:59.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:57:59 smithi043 ceph-mon[30062]: from='client.36254 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:59 smithi043 ceph-mon[41699]: pgmap v379: 356 pgs: 356 active+clean; 4.6 MiB data, 601 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:57:59.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:59 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:57:59.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:59 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:57:59.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:59 smithi043 ceph-mon[41699]: from='client.36254 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi043-81661-53"}]': finished 2024-03-21T12:57:59.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:59 smithi043 ceph-mon[41699]: osdmap e278: 8 total, 8 up, 8 in 2024-03-21T12:57:59.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:59 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/954503638' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:57:59.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:57:59 smithi043 ceph-mon[41699]: from='client.36254 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81661-53"}]: dispatch 2024-03-21T12:58:00.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:00 smithi155 ceph-mon[33846]: from='client.36254 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81661-53"}]': finished 2024-03-21T12:58:00.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:00 smithi155 ceph-mon[33846]: osdmap e279: 8 total, 8 up, 8 in 2024-03-21T12:58:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:00 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:00 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:00 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/572958402' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:00 smithi155 ceph-mon[33846]: from='client.26254 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:00 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/572958402' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:00 smithi155 ceph-mon[33846]: from='client.26254 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:00 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/572958402' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-81661-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:00 smithi155 ceph-mon[33846]: from='client.26254 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-81661-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:00.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:00 smithi043 ceph-mon[30062]: from='client.36254 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81661-53"}]': finished 2024-03-21T12:58:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:00 smithi043 ceph-mon[30062]: osdmap e279: 8 total, 8 up, 8 in 2024-03-21T12:58:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:00 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:00 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:00 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/572958402' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:00 smithi043 ceph-mon[30062]: from='client.26254 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:00 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/572958402' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:00 smithi043 ceph-mon[30062]: from='client.26254 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:00 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/572958402' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-81661-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:00 smithi043 ceph-mon[30062]: from='client.26254 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-81661-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:00.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:00 smithi043 ceph-mon[41699]: from='client.36254 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi043-81661-53"}]': finished 2024-03-21T12:58:00.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:00 smithi043 ceph-mon[41699]: osdmap e279: 8 total, 8 up, 8 in 2024-03-21T12:58:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:00 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:00 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:00 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/572958402' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:00 smithi043 ceph-mon[41699]: from='client.26254 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:00 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/572958402' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:00 smithi043 ceph-mon[41699]: from='client.26254 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:00 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/572958402' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-81661-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:00 smithi043 ceph-mon[41699]: from='client.26254 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-81661-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:01.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:01 smithi155 ceph-mon[33846]: pgmap v382: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T12:58:01.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:01 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:58:01.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:01 smithi155 ceph-mon[33846]: from='client.26254 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-81661-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:58:01.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:01 smithi155 ceph-mon[33846]: osdmap e280: 8 total, 8 up, 8 in 2024-03-21T12:58:01.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:01 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/572958402' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-81661-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:01.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:01 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:01.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:01 smithi155 ceph-mon[33846]: from='client.26254 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-81661-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:01.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:01 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:01.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:01 smithi043 ceph-mon[41699]: pgmap v382: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T12:58:01.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:01 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:58:01.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:01 smithi043 ceph-mon[41699]: from='client.26254 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-81661-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:58:01.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:01 smithi043 ceph-mon[41699]: osdmap e280: 8 total, 8 up, 8 in 2024-03-21T12:58:01.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:01 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/572958402' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-81661-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:01.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:01 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:01.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:01 smithi043 ceph-mon[41699]: from='client.26254 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-81661-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:01.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:01 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:01.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:01 smithi043 ceph-mon[30062]: pgmap v382: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T12:58:01.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:01 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:58:01.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:01 smithi043 ceph-mon[30062]: from='client.26254 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi043-81661-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:58:01.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:01 smithi043 ceph-mon[30062]: osdmap e280: 8 total, 8 up, 8 in 2024-03-21T12:58:01.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:01 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/572958402' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-81661-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:01.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:01 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:01.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:01 smithi043 ceph-mon[30062]: from='client.26254 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-81661-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:01.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:01 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:02.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:02 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-39", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:58:02.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:02 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-39"}]: dispatch 2024-03-21T12:58:02.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:02 smithi155 ceph-mon[33846]: osdmap e281: 8 total, 8 up, 8 in 2024-03-21T12:58:02.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:02 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-39"}]: dispatch 2024-03-21T12:58:02.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:02 smithi155 ceph-mon[33846]: from='client.26254 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-81661-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-81661-54"}]': finished 2024-03-21T12:58:02.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:02 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-39"}]': finished 2024-03-21T12:58:02.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:02 smithi155 ceph-mon[33846]: osdmap e282: 8 total, 8 up, 8 in 2024-03-21T12:58:02.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:02 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-39", "mode": "writeback"}]: dispatch 2024-03-21T12:58:02.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:02 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-39", "mode": "writeback"}]: dispatch 2024-03-21T12:58:02.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:02 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-39", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:58:02.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:02 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-39"}]: dispatch 2024-03-21T12:58:02.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:02 smithi043 ceph-mon[30062]: osdmap e281: 8 total, 8 up, 8 in 2024-03-21T12:58:02.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:02 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-39"}]: dispatch 2024-03-21T12:58:02.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:02 smithi043 ceph-mon[30062]: from='client.26254 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-81661-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-81661-54"}]': finished 2024-03-21T12:58:02.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:02 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-39"}]': finished 2024-03-21T12:58:02.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:02 smithi043 ceph-mon[30062]: osdmap e282: 8 total, 8 up, 8 in 2024-03-21T12:58:02.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:02 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-39", "mode": "writeback"}]: dispatch 2024-03-21T12:58:02.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:02 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-39", "mode": "writeback"}]: dispatch 2024-03-21T12:58:02.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:02 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-39", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:58:02.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:02 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-39"}]: dispatch 2024-03-21T12:58:02.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:02 smithi043 ceph-mon[41699]: osdmap e281: 8 total, 8 up, 8 in 2024-03-21T12:58:02.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:02 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-39"}]: dispatch 2024-03-21T12:58:02.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:02 smithi043 ceph-mon[41699]: from='client.26254 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi043-81661-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi043-81661-54"}]': finished 2024-03-21T12:58:02.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:02 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-39"}]': finished 2024-03-21T12:58:02.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:02 smithi043 ceph-mon[41699]: osdmap e282: 8 total, 8 up, 8 in 2024-03-21T12:58:02.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:02 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-39", "mode": "writeback"}]: dispatch 2024-03-21T12:58:02.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:02 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-39", "mode": "writeback"}]: dispatch 2024-03-21T12:58:03.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:03 smithi155 ceph-mon[33846]: pgmap v385: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T12:58:03.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:03 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:58:03.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:03 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-39", "mode": "writeback"}]': finished 2024-03-21T12:58:03.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:03 smithi155 ceph-mon[33846]: osdmap e283: 8 total, 8 up, 8 in 2024-03-21T12:58:03.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:03 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:03.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:03 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:03.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:03 smithi043 ceph-mon[30062]: pgmap v385: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T12:58:03.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:03 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:58:03.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:03 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-39", "mode": "writeback"}]': finished 2024-03-21T12:58:03.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:03 smithi043 ceph-mon[30062]: osdmap e283: 8 total, 8 up, 8 in 2024-03-21T12:58:03.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:03 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:03.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:03 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:03.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:03 smithi043 ceph-mon[41699]: pgmap v385: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T12:58:03.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:03 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:58:03.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:03 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-39", "mode": "writeback"}]': finished 2024-03-21T12:58:03.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:03 smithi043 ceph-mon[41699]: osdmap e283: 8 total, 8 up, 8 in 2024-03-21T12:58:03.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:03 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:03.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:03 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:04.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:04 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:58:04.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:04 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:58:04.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:04 smithi155 ceph-mon[33846]: osdmap e284: 8 total, 8 up, 8 in 2024-03-21T12:58:04.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/572958402' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:04.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:04 smithi155 ceph-mon[33846]: from='client.26254 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:04.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-39"}]: dispatch 2024-03-21T12:58:04.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:04 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-39"}]: dispatch 2024-03-21T12:58:04.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:04 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:58:04.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:04 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:58:04.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:04 smithi043 ceph-mon[30062]: osdmap e284: 8 total, 8 up, 8 in 2024-03-21T12:58:04.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/572958402' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:04.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:04 smithi043 ceph-mon[30062]: from='client.26254 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:04.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-39"}]: dispatch 2024-03-21T12:58:04.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:04 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-39"}]: dispatch 2024-03-21T12:58:04.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:04 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:58:04.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:04 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:58:04.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:04 smithi043 ceph-mon[41699]: osdmap e284: 8 total, 8 up, 8 in 2024-03-21T12:58:04.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/572958402' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:04.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:04 smithi043 ceph-mon[41699]: from='client.26254 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:04.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-39"}]: dispatch 2024-03-21T12:58:04.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:04 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-39"}]: dispatch 2024-03-21T12:58:05.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:05 smithi155 ceph-mon[33846]: pgmap v388: 364 pgs: 8 creating+peering, 356 active+clean; 4.6 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:05.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:05 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:58:05.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:05 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:58:05.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:05 smithi155 ceph-mon[33846]: from='client.26254 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81661-54"}]': finished 2024-03-21T12:58:05.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:05 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-39"}]': finished 2024-03-21T12:58:05.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:05 smithi155 ceph-mon[33846]: osdmap e285: 8 total, 8 up, 8 in 2024-03-21T12:58:05.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:05 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/572958402' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:05.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:05 smithi155 ceph-mon[33846]: from='client.26254 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:05 smithi043 ceph-mon[30062]: pgmap v388: 364 pgs: 8 creating+peering, 356 active+clean; 4.6 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:05 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:58:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:05 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:58:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:05 smithi043 ceph-mon[30062]: from='client.26254 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81661-54"}]': finished 2024-03-21T12:58:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:05 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-39"}]': finished 2024-03-21T12:58:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:05 smithi043 ceph-mon[30062]: osdmap e285: 8 total, 8 up, 8 in 2024-03-21T12:58:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:05 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/572958402' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:05 smithi043 ceph-mon[30062]: from='client.26254 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:05 smithi043 ceph-mon[41699]: pgmap v388: 364 pgs: 8 creating+peering, 356 active+clean; 4.6 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:05 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:58:05.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:05 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:58:05.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:05 smithi043 ceph-mon[41699]: from='client.26254 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi043-81661-54"}]': finished 2024-03-21T12:58:05.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:05 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-39"}]': finished 2024-03-21T12:58:05.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:05 smithi043 ceph-mon[41699]: osdmap e285: 8 total, 8 up, 8 in 2024-03-21T12:58:05.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:05 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/572958402' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:05.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:05 smithi043 ceph-mon[41699]: from='client.26254 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81661-54"}]: dispatch 2024-03-21T12:58:06.398 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-03-21T12:58:06.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-03-21T12:58:06.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-03-21T12:58:06.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: seed 81880 2024-03-21T12:58:06.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-03-21T12:58:06.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (383 ms) 2024-03-21T12:58:06.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-03-21T12:58:06.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (16024 ms) 2024-03-21T12:58:06.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-03-21T12:58:06.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (46 ms) 2024-03-21T12:58:06.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (16453 ms total) 2024-03-21T12:58:06.399 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 2024-03-21T12:58:06.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-03-21T12:58:06.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-03-21T12:58:06.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (7091 ms) 2024-03-21T12:58:06.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-03-21T12:58:06.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (7904 ms) 2024-03-21T12:58:06.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-03-21T12:58:06.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (9914 ms) 2024-03-21T12:58:06.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-03-21T12:58:06.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: my_snaps [3] 2024-03-21T12:58:06.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: my_snaps [4,3] 2024-03-21T12:58:06.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: my_snaps [5,4,3] 2024-03-21T12:58:06.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: my_snaps [6,5,4,3] 2024-03-21T12:58:06.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: promoting some heads 2024-03-21T12:58:06.400 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: promoting from clones for snap 6 2024-03-21T12:58:06.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: promoting from clones for snap 5 2024-03-21T12:58:06.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: promoting from clones for snap 4 2024-03-21T12:58:06.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: promoting from clones for snap 3 2024-03-21T12:58:06.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: waiting for scrubs... 2024-03-21T12:58:06.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: done waiting 2024-03-21T12:58:06.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (46026 ms) 2024-03-21T12:58:06.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-03-21T12:58:06.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (9997 ms) 2024-03-21T12:58:06.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-03-21T12:58:06.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (8000 ms) 2024-03-21T12:58:06.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-03-21T12:58:06.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8031 ms) 2024-03-21T12:58:06.401 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-03-21T12:58:06.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8051 ms) 2024-03-21T12:58:06.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-03-21T12:58:06.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (10276 ms) 2024-03-21T12:58:06.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-03-21T12:58:06.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (8654 ms) 2024-03-21T12:58:06.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-03-21T12:58:06.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10293 ms) 2024-03-21T12:58:06.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-03-21T12:58:06.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (11730 ms) 2024-03-21T12:58:06.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-03-21T12:58:06.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (7975 ms) 2024-03-21T12:58:06.402 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-03-21T12:58:06.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (8297 ms) 2024-03-21T12:58:06.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-03-21T12:58:06.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (12785 ms) 2024-03-21T12:58:06.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-03-21T12:58:06.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7950 ms) 2024-03-21T12:58:06.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-03-21T12:58:06.403 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (8022 ms) 2024-03-21T12:58:07.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:07 smithi155 ceph-mon[33846]: pgmap v391: 356 pgs: 356 active+clean; 4.6 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:07.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:07 smithi155 ceph-mon[33846]: from='client.26254 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81661-54"}]': finished 2024-03-21T12:58:07.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:07 smithi155 ceph-mon[33846]: osdmap e286: 8 total, 8 up, 8 in 2024-03-21T12:58:07.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:07 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2508440685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:07.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:07 smithi155 ceph-mon[33846]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:07.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:07 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2508440685' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:07.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:07 smithi155 ceph-mon[33846]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:07.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:07 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2508440685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-81661-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:07.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:07 smithi155 ceph-mon[33846]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-81661-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:07 smithi043 ceph-mon[30062]: pgmap v391: 356 pgs: 356 active+clean; 4.6 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:07 smithi043 ceph-mon[30062]: from='client.26254 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81661-54"}]': finished 2024-03-21T12:58:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:07 smithi043 ceph-mon[30062]: osdmap e286: 8 total, 8 up, 8 in 2024-03-21T12:58:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:07 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2508440685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:07 smithi043 ceph-mon[30062]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:07 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2508440685' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:07 smithi043 ceph-mon[30062]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:07 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2508440685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-81661-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:07 smithi043 ceph-mon[30062]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-81661-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:07.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:07 smithi043 ceph-mon[41699]: pgmap v391: 356 pgs: 356 active+clean; 4.6 MiB data, 603 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:07.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:07 smithi043 ceph-mon[41699]: from='client.26254 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi043-81661-54"}]': finished 2024-03-21T12:58:07.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:07 smithi043 ceph-mon[41699]: osdmap e286: 8 total, 8 up, 8 in 2024-03-21T12:58:07.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:07 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2508440685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:07.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:07 smithi043 ceph-mon[41699]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:07.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:07 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2508440685' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:07.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:07 smithi043 ceph-mon[41699]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:07.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:07 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2508440685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-81661-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:07.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:07 smithi043 ceph-mon[41699]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-81661-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:08.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:58:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:58:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:58:08.666 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:58:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:58:08.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:08 smithi155 ceph-mon[33846]: from='client.36266 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-81661-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:58:08.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:08 smithi155 ceph-mon[33846]: osdmap e287: 8 total, 8 up, 8 in 2024-03-21T12:58:08.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:08 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2508440685' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-81661-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:08.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:08 smithi155 ceph-mon[33846]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-81661-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:08.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:08 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:08.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:08 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:08.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:08 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:58:08.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:08 smithi043 ceph-mon[30062]: from='client.36266 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-81661-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:58:08.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:08 smithi043 ceph-mon[30062]: osdmap e287: 8 total, 8 up, 8 in 2024-03-21T12:58:08.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:08 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2508440685' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-81661-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:08.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:08 smithi043 ceph-mon[30062]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-81661-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:08.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:08 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:08.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:08 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:08.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:08 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:58:08.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:08 smithi043 ceph-mon[41699]: from='client.36266 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi043-81661-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:58:08.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:08 smithi043 ceph-mon[41699]: osdmap e287: 8 total, 8 up, 8 in 2024-03-21T12:58:08.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:08 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2508440685' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-81661-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:08.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:08 smithi043 ceph-mon[41699]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-81661-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:08.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:08 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:08.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:08 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:08.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:08 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:58:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:09 smithi155 ceph-mon[33846]: pgmap v394: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 603 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:58:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:09 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:58:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:09 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:09 smithi155 ceph-mon[33846]: osdmap e288: 8 total, 8 up, 8 in 2024-03-21T12:58:09.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:09 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:09.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:09 smithi155 ceph-mon[33846]: from='client.36266 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-81661-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-81661-55"}]': finished 2024-03-21T12:58:09.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:09 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-41", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:58:09.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:09 smithi155 ceph-mon[33846]: osdmap e289: 8 total, 8 up, 8 in 2024-03-21T12:58:09.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:09 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T12:58:09.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:09 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T12:58:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:09 smithi043 ceph-mon[30062]: pgmap v394: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 603 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:58:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:09 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:58:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:09 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:09 smithi043 ceph-mon[30062]: osdmap e288: 8 total, 8 up, 8 in 2024-03-21T12:58:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:09 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:09 smithi043 ceph-mon[30062]: from='client.36266 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-81661-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-81661-55"}]': finished 2024-03-21T12:58:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:09 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-41", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:58:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:09 smithi043 ceph-mon[30062]: osdmap e289: 8 total, 8 up, 8 in 2024-03-21T12:58:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:09 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T12:58:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:09 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T12:58:09.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:09 smithi043 ceph-mon[41699]: pgmap v394: 356 pgs: 32 unknown, 324 active+clean; 4.6 MiB data, 603 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:58:09.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:09 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:58:09.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:09 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:09.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:09 smithi043 ceph-mon[41699]: osdmap e288: 8 total, 8 up, 8 in 2024-03-21T12:58:09.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:09 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:09.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:09 smithi043 ceph-mon[41699]: from='client.36266 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi043-81661-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi043-81661-55"}]': finished 2024-03-21T12:58:09.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:09 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-41", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:58:09.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:09 smithi043 ceph-mon[41699]: osdmap e289: 8 total, 8 up, 8 in 2024-03-21T12:58:09.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:09 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T12:58:09.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:09 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T12:58:11.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:11 smithi155 ceph-mon[33846]: pgmap v397: 364 pgs: 8 unknown, 32 creating+peering, 324 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-21T12:58:11.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:11 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_count","val": "2"}]': finished 2024-03-21T12:58:11.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:11 smithi155 ceph-mon[33846]: osdmap e290: 8 total, 8 up, 8 in 2024-03-21T12:58:11.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:11 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T12:58:11.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:11 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T12:58:11.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:11 smithi043 ceph-mon[30062]: pgmap v397: 364 pgs: 8 unknown, 32 creating+peering, 324 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-21T12:58:11.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:11 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_count","val": "2"}]': finished 2024-03-21T12:58:11.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:11 smithi043 ceph-mon[30062]: osdmap e290: 8 total, 8 up, 8 in 2024-03-21T12:58:11.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:11 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T12:58:11.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:11 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T12:58:11.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:11 smithi043 ceph-mon[41699]: pgmap v397: 364 pgs: 8 unknown, 32 creating+peering, 324 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-21T12:58:11.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:11 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_count","val": "2"}]': finished 2024-03-21T12:58:11.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:11 smithi043 ceph-mon[41699]: osdmap e290: 8 total, 8 up, 8 in 2024-03-21T12:58:11.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:11 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T12:58:11.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:11 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T12:58:12.565 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetrted 0x558552526210 2024-03-21T12:58:12.565 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525266c0 2024-03-21T12:58:12.565 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552526810 2024-03-21T12:58:12.565 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552526920 2024-03-21T12:58:12.565 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552526a50 2024-03-21T12:58:12.565 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552527170 2024-03-21T12:58:12.565 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552526cc0 2024-03-21T12:58:12.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552526df0 2024-03-21T12:58:12.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552526f20 2024-03-21T12:58:12.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525272a0 2024-03-21T12:58:12.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552527750 2024-03-21T12:58:12.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552527880 2024-03-21T12:58:12.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525241f0 2024-03-21T12:58:12.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552524320 2024-03-21T12:58:12.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552526340 2024-03-21T12:58:12.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552524450 2024-03-21T12:58:12.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552527e60 2024-03-21T12:58:12.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525279b0 2024-03-21T12:58:12.566 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552527b00 2024-03-21T12:58:12.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552527c70 2024-03-21T12:58:12.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552527f90 2024-03-21T12:58:12.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552528140 2024-03-21T12:58:12.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552528720 2024-03-21T12:58:12.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552528270 2024-03-21T12:58:12.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525273b0 2024-03-21T12:58:12.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525274e0 2024-03-21T12:58:12.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552527610 2024-03-21T12:58:12.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552528d00 2024-03-21T12:58:12.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552528850 2024-03-21T12:58:12.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552528980 2024-03-21T12:58:12.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552528ab0 2024-03-21T12:58:12.567 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552528e30 2024-03-21T12:58:12.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552529300 2024-03-21T12:58:12.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552529430 2024-03-21T12:58:12.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552528380 2024-03-21T12:58:12.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525284b0 2024-03-21T12:58:12.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525285e0 2024-03-21T12:58:12.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552529560 2024-03-21T12:58:12.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552529a50 2024-03-21T12:58:12.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552529b60 2024-03-21T12:58:12.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552528f60 2024-03-21T12:58:12.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552529070 2024-03-21T12:58:12.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552528be0 2024-03-21T12:58:12.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525291a0 2024-03-21T12:58:12.568 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252a120 2024-03-21T12:58:12.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552529c70 2024-03-21T12:58:12.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552529dc0 2024-03-21T12:58:12.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552529f10 2024-03-21T12:58:12.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252a270 2024-03-21T12:58:12.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252a720 2024-03-21T12:58:12.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252a890 2024-03-21T12:58:12.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552529670 2024-03-21T12:58:12.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525297a0 2024-03-21T12:58:12.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525298d0 2024-03-21T12:58:12.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252a9e0 2024-03-21T12:58:12.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252ae90 2024-03-21T12:58:12.569 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252afe0 2024-03-21T12:58:12.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252a380 2024-03-21T12:58:12.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252a4d0 2024-03-21T12:58:12.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252a5e0 2024-03-21T12:58:12.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252b110 2024-03-21T12:58:12.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252b5e0 2024-03-21T12:58:12.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252b730 2024-03-21T12:58:12.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252b220 2024-03-21T12:58:12.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252b370 2024-03-21T12:58:12.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252b4c0 2024-03-21T12:58:12.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252b880 2024-03-21T12:58:12.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252bd50 2024-03-21T12:58:12.570 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252be80 2024-03-21T12:58:12.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252aaf0 2024-03-21T12:58:12.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252ac20 2024-03-21T12:58:12.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252ad50 2024-03-21T12:58:12.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252bfd0 2024-03-21T12:58:12.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252c4e0 2024-03-21T12:58:12.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252c630 2024-03-21T12:58:12.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252b9b0 2024-03-21T12:58:12.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252bae0 2024-03-21T12:58:12.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252bbf0 2024-03-21T12:58:12.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252c740 2024-03-21T12:58:12.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252cbf0 2024-03-21T12:58:12.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252cd00 2024-03-21T12:58:12.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252d300 2024-03-21T12:58:12.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:12 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_period","val": "600"}]': finished 2024-03-21T12:58:12.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:12 smithi155 ceph-mon[33846]: osdmap e291: 8 total, 8 up, 8 in 2024-03-21T12:58:12.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:12 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T12:58:12.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:12 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2508440685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:12.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:12 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T12:58:12.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:12 smithi155 ceph-mon[33846]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:12.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:12 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_period","val": "600"}]': finished 2024-03-21T12:58:12.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:12 smithi043 ceph-mon[30062]: osdmap e291: 8 total, 8 up, 8 in 2024-03-21T12:58:12.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:12 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T12:58:12.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:12 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2508440685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:12.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:12 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T12:58:12.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:12 smithi043 ceph-mon[30062]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:12.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:12 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_period","val": "600"}]': finished 2024-03-21T12:58:12.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:12 smithi043 ceph-mon[41699]: osdmap e291: 8 total, 8 up, 8 in 2024-03-21T12:58:12.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:12 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T12:58:12.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:12 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2508440685' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:12.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:12 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T12:58:12.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:12 smithi043 ceph-mon[41699]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:13.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:13 smithi155 ceph-mon[33846]: pgmap v400: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-03-21T12:58:13.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:13 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:58:13.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:13 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-21T12:58:13.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:13 smithi155 ceph-mon[33846]: from='client.36266 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81661-55"}]': finished 2024-03-21T12:58:13.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:13 smithi155 ceph-mon[33846]: osdmap e292: 8 total, 8 up, 8 in 2024-03-21T12:58:13.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:13 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2508440685' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:13.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:13 smithi155 ceph-mon[33846]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:13.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:13 smithi043 ceph-mon[30062]: pgmap v400: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-03-21T12:58:13.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:13 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:58:13.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:13 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-21T12:58:13.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:13 smithi043 ceph-mon[30062]: from='client.36266 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81661-55"}]': finished 2024-03-21T12:58:13.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:13 smithi043 ceph-mon[30062]: osdmap e292: 8 total, 8 up, 8 in 2024-03-21T12:58:13.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:13 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2508440685' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:13.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:13 smithi043 ceph-mon[30062]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:13.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:13 smithi043 ceph-mon[41699]: pgmap v400: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 615 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-03-21T12:58:13.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:13 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:58:13.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:13 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-21T12:58:13.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:13 smithi043 ceph-mon[41699]: from='client.36266 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi043-81661-55"}]': finished 2024-03-21T12:58:13.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:13 smithi043 ceph-mon[41699]: osdmap e292: 8 total, 8 up, 8 in 2024-03-21T12:58:13.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:13 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2508440685' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:13.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:13 smithi043 ceph-mon[41699]: from='client.36266 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81661-55"}]: dispatch 2024-03-21T12:58:14.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:14 smithi155 ceph-mon[33846]: from='client.36266 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81661-55"}]': finished 2024-03-21T12:58:14.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:14 smithi155 ceph-mon[33846]: osdmap e293: 8 total, 8 up, 8 in 2024-03-21T12:58:14.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:14 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1466731409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:14.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:14 smithi155 ceph-mon[33846]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:14.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:14 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1466731409' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:14.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:14 smithi155 ceph-mon[33846]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:14.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:14 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1466731409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-81661-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:14.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:14 smithi155 ceph-mon[33846]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-81661-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:14.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:14 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:14.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:14 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:14.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:14 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-41"}]: dispatch 2024-03-21T12:58:14.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:14 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-41"}]: dispatch 2024-03-21T12:58:14.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:14 smithi043 ceph-mon[30062]: from='client.36266 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81661-55"}]': finished 2024-03-21T12:58:14.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:14 smithi043 ceph-mon[30062]: osdmap e293: 8 total, 8 up, 8 in 2024-03-21T12:58:14.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:14 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1466731409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:14.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:14 smithi043 ceph-mon[30062]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:14.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:14 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1466731409' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:14.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:14 smithi043 ceph-mon[30062]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:14.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:14 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1466731409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-81661-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:14.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:14 smithi043 ceph-mon[30062]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-81661-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:14.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:14 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:14.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:14 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:14.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:14 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-41"}]: dispatch 2024-03-21T12:58:14.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:14 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-41"}]: dispatch 2024-03-21T12:58:14.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:14 smithi043 ceph-mon[41699]: from='client.36266 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi043-81661-55"}]': finished 2024-03-21T12:58:14.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:14 smithi043 ceph-mon[41699]: osdmap e293: 8 total, 8 up, 8 in 2024-03-21T12:58:14.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:14 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1466731409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:14.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:14 smithi043 ceph-mon[41699]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:14.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:14 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1466731409' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:14.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:14 smithi043 ceph-mon[41699]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:14.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:14 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1466731409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-81661-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:14.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:14 smithi043 ceph-mon[41699]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-81661-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:14.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:14 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:14.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:14 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:14.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:14 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-41"}]: dispatch 2024-03-21T12:58:14.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:14 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-41"}]: dispatch 2024-03-21T12:58:15.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:15 smithi155 ceph-mon[33846]: pgmap v403: 356 pgs: 356 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:15.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:15 smithi155 ceph-mon[33846]: from='client.36272 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-81661-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:58:15.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:15 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-41"}]': finished 2024-03-21T12:58:15.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:15 smithi155 ceph-mon[33846]: osdmap e294: 8 total, 8 up, 8 in 2024-03-21T12:58:15.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:15 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1466731409' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-81661-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:15.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:15 smithi155 ceph-mon[33846]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-81661-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:15.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:15 smithi043 ceph-mon[30062]: pgmap v403: 356 pgs: 356 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:15.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:15 smithi043 ceph-mon[30062]: from='client.36272 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-81661-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:58:15.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:15 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-41"}]': finished 2024-03-21T12:58:15.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:15 smithi043 ceph-mon[30062]: osdmap e294: 8 total, 8 up, 8 in 2024-03-21T12:58:15.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:15 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1466731409' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-81661-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:15.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:15 smithi043 ceph-mon[30062]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-81661-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:15.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:15 smithi043 ceph-mon[41699]: pgmap v403: 356 pgs: 356 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:15.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:15 smithi043 ceph-mon[41699]: from='client.36272 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi043-81661-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:58:15.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:15 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-41"}]': finished 2024-03-21T12:58:15.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:15 smithi043 ceph-mon[41699]: osdmap e294: 8 total, 8 up, 8 in 2024-03-21T12:58:15.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:15 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1466731409' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-81661-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:15.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:15 smithi043 ceph-mon[41699]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-81661-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:16.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:16 smithi155 ceph-mon[33846]: osdmap e295: 8 total, 8 up, 8 in 2024-03-21T12:58:16.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:16 smithi043 ceph-mon[30062]: osdmap e295: 8 total, 8 up, 8 in 2024-03-21T12:58:16.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:16 smithi043 ceph-mon[41699]: osdmap e295: 8 total, 8 up, 8 in 2024-03-21T12:58:17.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:17 smithi155 ceph-mon[33846]: pgmap v406: 324 pgs: 324 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:17.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:17 smithi155 ceph-mon[33846]: from='client.36272 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-81661-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-81661-56"}]': finished 2024-03-21T12:58:17.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:17 smithi155 ceph-mon[33846]: osdmap e296: 8 total, 8 up, 8 in 2024-03-21T12:58:17.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:17 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:17.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:17 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:17.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:17 smithi043 ceph-mon[30062]: pgmap v406: 324 pgs: 324 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:17.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:17 smithi043 ceph-mon[30062]: from='client.36272 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-81661-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-81661-56"}]': finished 2024-03-21T12:58:17.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:17 smithi043 ceph-mon[30062]: osdmap e296: 8 total, 8 up, 8 in 2024-03-21T12:58:17.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:17 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:17.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:17 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:17.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:17 smithi043 ceph-mon[41699]: pgmap v406: 324 pgs: 324 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:17.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:17 smithi043 ceph-mon[41699]: from='client.36272 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi043-81661-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi043-81661-56"}]': finished 2024-03-21T12:58:17.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:17 smithi043 ceph-mon[41699]: osdmap e296: 8 total, 8 up, 8 in 2024-03-21T12:58:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:17 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:17.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:17 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:18.455 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:58:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:58:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:58:18.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:58:18 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:58:18.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:18 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:58:18.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:18 smithi155 ceph-mon[33846]: osdmap e297: 8 total, 8 up, 8 in 2024-03-21T12:58:18.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:18 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi043-81880-6","var": "pg_num","format": "json"}]: dispatch 2024-03-21T12:58:18.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:18 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:18.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:18 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:18.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:18 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:58:18.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:18 smithi043 ceph-mon[30062]: osdmap e297: 8 total, 8 up, 8 in 2024-03-21T12:58:18.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:18 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi043-81880-6","var": "pg_num","format": "json"}]: dispatch 2024-03-21T12:58:18.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:18 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:18.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:18 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:18.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:18 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:58:18.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:18 smithi043 ceph-mon[41699]: osdmap e297: 8 total, 8 up, 8 in 2024-03-21T12:58:18.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:18 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi043-81880-6","var": "pg_num","format": "json"}]: dispatch 2024-03-21T12:58:18.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:18 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:18.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:18 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:19.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:19 smithi155 ceph-mon[33846]: pgmap v409: 364 pgs: 40 unknown, 324 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:58:19.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:19 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:58:19.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:19 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:58:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:19 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-43", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:58:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:19 smithi155 ceph-mon[33846]: osdmap e298: 8 total, 8 up, 8 in 2024-03-21T12:58:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:19 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-21T12:58:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:19 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-21T12:58:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:19 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1466731409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:19 smithi155 ceph-mon[33846]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:19 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:58:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:19 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:58:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:19 smithi043 ceph-mon[30062]: pgmap v409: 364 pgs: 40 unknown, 324 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:58:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:19 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:58:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:19 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:58:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:19 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-43", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:58:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:19 smithi043 ceph-mon[30062]: osdmap e298: 8 total, 8 up, 8 in 2024-03-21T12:58:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:19 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-21T12:58:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:19 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-21T12:58:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:19 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1466731409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:19 smithi043 ceph-mon[30062]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:19 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:58:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:19 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:58:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:19 smithi043 ceph-mon[41699]: pgmap v409: 364 pgs: 40 unknown, 324 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:58:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:19 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:58:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:19 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:58:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:19 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-43", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:58:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:19 smithi043 ceph-mon[41699]: osdmap e298: 8 total, 8 up, 8 in 2024-03-21T12:58:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:19 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-21T12:58:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:19 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-21T12:58:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:19 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1466731409' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:19 smithi043 ceph-mon[41699]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:19 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:58:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:19 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:58:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:20 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_count","val": "8"}]': finished 2024-03-21T12:58:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:20 smithi043 ceph-mon[30062]: from='client.36272 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81661-56"}]': finished 2024-03-21T12:58:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:20 smithi043 ceph-mon[30062]: osdmap e299: 8 total, 8 up, 8 in 2024-03-21T12:58:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:20 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T12:58:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:20 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1466731409' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:20 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T12:58:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:20 smithi043 ceph-mon[30062]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:20.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:20 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_count","val": "8"}]': finished 2024-03-21T12:58:20.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:20 smithi043 ceph-mon[41699]: from='client.36272 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81661-56"}]': finished 2024-03-21T12:58:20.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:20 smithi043 ceph-mon[41699]: osdmap e299: 8 total, 8 up, 8 in 2024-03-21T12:58:20.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:20 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T12:58:20.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:20 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1466731409' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:20.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:20 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T12:58:20.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:20 smithi043 ceph-mon[41699]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:20.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:20 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_count","val": "8"}]': finished 2024-03-21T12:58:20.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:20 smithi155 ceph-mon[33846]: from='client.36272 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi043-81661-56"}]': finished 2024-03-21T12:58:20.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:20 smithi155 ceph-mon[33846]: osdmap e299: 8 total, 8 up, 8 in 2024-03-21T12:58:20.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:20 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T12:58:20.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:20 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1466731409' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:20.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:20 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T12:58:20.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:20 smithi155 ceph-mon[33846]: from='client.36272 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81661-56"}]: dispatch 2024-03-21T12:58:21.712 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc:Read 2024-03-21T12:58:21.712 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: hmm, no HitSet yet 2024-03-21T12:58:21.712 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: ok, hit_set contains 262:602f83fe:::foo:head 2024-03-21T12:58:21.712 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9295 ms) 2024-03-21T12:58:21.712 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-03-21T12:58:21.712 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg_num = 32 2024-03-21T12:58:21.712 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 0 ls 1711025902,0 2024-03-21T12:58:21.712 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 1 ls 1711025902,0 2024-03-21T12:58:21.712 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 2 ls 1711025902,0 2024-03-21T12:58:21.712 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 3 ls 1711025902,0 2024-03-21T12:58:21.712 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 4 ls 1711025902,0 2024-03-21T12:58:21.712 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 5 ls 1711025902,0 2024-03-21T12:58:21.713 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 6 ls 1711025902,0 2024-03-21T12:58:21.713 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 7 ls 1711025902,0 2024-03-21T12:58:21.713 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 8 ls 1711025902,0 2024-03-21T12:58:21.713 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 9 ls 1711025902,0 2024-03-21T12:58:21.713 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 10 ls 1711025902,0 2024-03-21T12:58:21.713 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 11 ls 1711025902,0 2024-03-21T12:58:21.713 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 12 ls 1711025902,0 2024-03-21T12:58:21.713 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 13 ls 1711025902,0 2024-03-21T12:58:21.713 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 14 ls 1711025902,0 2024-03-21T12:58:21.713 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 15 ls 1711025902,0 2024-03-21T12:58:21.713 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 16 ls 1711025902,0 2024-03-21T12:58:21.713 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 17 ls 1711025902,0 2024-03-21T12:58:21.713 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 18 ls 1711025902,0 2024-03-21T12:58:21.713 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 19 ls 1711025902,0 2024-03-21T12:58:21.713 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 20 ls 1711025902,0 2024-03-21T12:58:21.714 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 21 ls 1711025902,0 2024-03-21T12:58:21.714 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 22 ls 1711025902,0 2024-03-21T12:58:21.714 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 23 ls 1711025902,0 2024-03-21T12:58:21.714 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 24 ls 1711025902,0 2024-03-21T12:58:21.714 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 25 ls 1711025902,0 2024-03-21T12:58:21.714 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 26 ls 1711025902,0 2024-03-21T12:58:21.714 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 27 ls 1711025902,0 2024-03-21T12:58:21.714 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 28 ls 1711025902,0 2024-03-21T12:58:21.714 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 29 ls 1711025902,0 2024-03-21T12:58:21.714 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 30 ls 1711025902,0 2024-03-21T12:58:21.714 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg 31 ls 1711025902,0 2024-03-21T12:58:21.714 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: pg_num = 32 2024-03-21T12:58:21.714 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:6cac518f:::0:head 2024-03-21T12:58:21.714 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:02547ec2:::1:head 2024-03-21T12:58:21.714 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:f905c69b:::2:head 2024-03-21T12:58:21.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:cfc208b3:::3:head 2024-03-21T12:58:21.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:d83876eb:::4:head 2024-03-21T12:58:21.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:b29083e3:::5:head 2024-03-21T12:58:21.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:c4fdafeb:::6:head 2024-03-21T12:58:21.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:5c6b0b28:::7:head 2024-03-21T12:58:21.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:bd63b0f1:::8:head 2024-03-21T12:58:21.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:e960b815:::9:head 2024-03-21T12:58:21.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:52ea6a34:::10:head 2024-03-21T12:58:21.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:89d3ae78:::11:head 2024-03-21T12:58:21.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:de5d7c5f:::12:head 2024-03-21T12:58:21.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:566253c9:::13:head 2024-03-21T12:58:21.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:62a1935d:::14:head 2024-03-21T12:58:21.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:863748b0:::15:head 2024-03-21T12:58:21.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:3958e169:::16:head 2024-03-21T12:58:21.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:4d4dabf9:::17:head 2024-03-21T12:58:21.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:8391935d:::18:head 2024-03-21T12:58:21.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:28883081:::19:head 2024-03-21T12:58:21.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:69259c59:::20:head 2024-03-21T12:58:21.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:4bdb80b7:::21:head 2024-03-21T12:58:21.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:a11c5d71:::22:head 2024-03-21T12:58:21.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:271af37b:::23:head 2024-03-21T12:58:21.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:95b121be:::24:head 2024-03-21T12:58:21.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:58d1031b:::25:head 2024-03-21T12:58:21.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:0a050783:::26:head 2024-03-21T12:58:21.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:c709704c:::27:head 2024-03-21T12:58:21.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:cbe56eaf:::28:head 2024-03-21T12:58:21.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:86b4b162:::29:head 2024-03-21T12:58:21.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:70d89383:::30:head 2024-03-21T12:58:21.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:dd450c7c:::31:head 2024-03-21T12:58:21.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:6d5729b1:::32:head 2024-03-21T12:58:21.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:c388f3fb:::33:head 2024-03-21T12:58:21.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:56cfea31:::34:head 2024-03-21T12:58:21.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:9dbc1bf7:::35:head 2024-03-21T12:58:21.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:40b74ccd:::36:head 2024-03-21T12:58:21.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:4d5aaf42:::37:head 2024-03-21T12:58:21.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:920f362c:::38:head 2024-03-21T12:58:21.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:6cc53222:::39:head 2024-03-21T12:58:21.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:9cad833f:::40:head 2024-03-21T12:58:21.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:1ea84d41:::41:head 2024-03-21T12:58:21.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:c4480ef6:::42:head 2024-03-21T12:58:21.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:a694361e:::43:head 2024-03-21T12:58:21.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:d1bd33e9:::44:head 2024-03-21T12:58:21.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:ddc2cd5d:::45:head 2024-03-21T12:58:21.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:2b782207:::46:head 2024-03-21T12:58:21.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:7b187fca:::47:head 2024-03-21T12:58:21.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:90ecdf6f:::48:head 2024-03-21T12:58:21.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:a5ed95fe:::49:head 2024-03-21T12:58:21.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:ea0eaa55:::50:head 2024-03-21T12:58:21.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:f33ef17b:::51:head 2024-03-21T12:58:21.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:a0d1b2f6:::52:head 2024-03-21T12:58:21.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:60c5229e:::53:head 2024-03-21T12:58:21.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:edcbc575:::54:head 2024-03-21T12:58:21.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:102cf253:::55:head 2024-03-21T12:58:21.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:efb7fb0b:::56:head 2024-03-21T12:58:21.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:50d0a326:::57:head 2024-03-21T12:58:21.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:d4dc5daf:::58:head 2024-03-21T12:58:21.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:3a130462:::59:head 2024-03-21T12:58:21.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:ec87ed71:::60:head 2024-03-21T12:58:21.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:d5bc9454:::61:head 2024-03-21T12:58:21.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:3ddfe313:::62:head 2024-03-21T12:58:21.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:7c2816b9:::63:head 2024-03-21T12:58:21.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:47e00e4d:::64:head 2024-03-21T12:58:21.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:c6410c18:::65:head 2024-03-21T12:58:21.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:b48ed237:::66:head 2024-03-21T12:58:21.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:cd63ad31:::67:head 2024-03-21T12:58:21.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:b179e92b:::68:head 2024-03-21T12:58:21.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:0d9f741a:::69:head 2024-03-21T12:58:21.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:6d3352ae:::70:head 2024-03-21T12:58:21.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:c6d5c19e:::71:head 2024-03-21T12:58:21.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:bc4729c3:::72:head 2024-03-21T12:58:21.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:77e930b9:::73:head 2024-03-21T12:58:21.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:0abeecfd:::74:head 2024-03-21T12:58:21.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:b7c37e15:::75:head 2024-03-21T12:58:21.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:b6378398:::76:head 2024-03-21T12:58:21.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:02bd68de:::77:head 2024-03-21T12:58:21.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:cc795d2d:::78:head 2024-03-21T12:58:21.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:630d4fea:::79:head 2024-03-21T12:58:21.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:e0d29ef5:::80:head 2024-03-21T12:58:21.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:fd6f13d2:::81:head 2024-03-21T12:58:21.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:606461d5:::82:head 2024-03-21T12:58:21.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:eadbdc43:::83:head 2024-03-21T12:58:21.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:8761d0bb:::84:head 2024-03-21T12:58:21.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:9ef0186f:::85:head 2024-03-21T12:58:21.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:e0d41294:::86:head 2024-03-21T12:58:21.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:961de695:::87:head 2024-03-21T12:58:21.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:1423148f:::88:head 2024-03-21T12:58:21.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:633a8fa2:::89:head 2024-03-21T12:58:21.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:a8653809:::90:head 2024-03-21T12:58:21.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:3dac8b33:::91:head 2024-03-21T12:58:21.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:35aad435:::92:head 2024-03-21T12:58:21.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:f6dcc343:::93:head 2024-03-21T12:58:21.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:dbbdad87:::94:head 2024-03-21T12:58:21.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:1cb48ce0:::95:head 2024-03-21T12:58:21.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:03cd461c:::96:head 2024-03-21T12:58:21.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:17a4ea99:::97:head 2024-03-21T12:58:21.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:9993c9a7:::98:head 2024-03-21T12:58:21.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:6394211c:::99:head 2024-03-21T12:58:21.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:94c7ae57:::100:head 2024-03-21T12:58:21.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:6fdee5bb:::101:head 2024-03-21T12:58:21.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:9a477fd1:::102:head 2024-03-21T12:58:21.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:eb850916:::103:head 2024-03-21T12:58:21.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:affc56b9:::104:head 2024-03-21T12:58:21.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:b42dc814:::105:head 2024-03-21T12:58:21.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:f319f8f0:::106:head 2024-03-21T12:58:21.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:9a40b9de:::107:head 2024-03-21T12:58:21.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:8b524f28:::108:head 2024-03-21T12:58:21.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:e3de589f:::109:head 2024-03-21T12:58:21.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:90f90a5b:::110:head 2024-03-21T12:58:21.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:a7b4f1d7:::111:head 2024-03-21T12:58:21.722 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:af51766e:::112:head 2024-03-21T12:58:21.722 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:b6f90bd1:::113:head 2024-03-21T12:58:21.722 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:e0261208:::114:head 2024-03-21T12:58:21.722 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:c9569ef7:::115:head 2024-03-21T12:58:21.722 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:61bebe50:::116:head 2024-03-21T12:58:21.722 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:fe93412b:::117:head 2024-03-21T12:58:21.722 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:d3d38bee:::118:head 2024-03-21T12:58:21.722 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:3100ba0c:::119:head 2024-03-21T12:58:21.722 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:d0560ada:::120:head 2024-03-21T12:58:21.722 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:f0ea8b35:::121:head 2024-03-21T12:58:21.722 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:766f231a:::122:head 2024-03-21T12:58:21.722 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:a07a2582:::123:head 2024-03-21T12:58:21.722 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:bd7c6b3a:::124:head 2024-03-21T12:58:21.722 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:fb2ddaff:::125:head 2024-03-21T12:58:21.722 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:4408e1fe:::126:head 2024-03-21T12:58:21.723 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:ee1df7a7:::127:head 2024-03-21T12:58:21.723 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:c3002909:::128:head 2024-03-21T12:58:21.723 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:4f48ffa9:::129:head 2024-03-21T12:58:21.723 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:edf38733:::130:head 2024-03-21T12:58:21.723 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:c08425c0:::131:head 2024-03-21T12:58:21.723 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:5f902d98:::132:head 2024-03-21T12:58:21.723 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:41ea2c93:::133:head 2024-03-21T12:58:21.723 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:813cee13:::134:head 2024-03-21T12:58:21.723 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:0131818d:::135:head 2024-03-21T12:58:21.723 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:26ba5a85:::136:head 2024-03-21T12:58:21.723 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:381b8a5a:::137:head 2024-03-21T12:58:21.723 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:28797e47:::138:head 2024-03-21T12:58:21.723 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:bfca7f22:::139:head 2024-03-21T12:58:21.723 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:36807075:::140:head 2024-03-21T12:58:21.723 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:80b03975:::141:head 2024-03-21T12:58:21.723 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:5c15709b:::142:head 2024-03-21T12:58:21.723 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:f39ea15e:::143:head 2024-03-21T12:58:21.723 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:ea992956:::144:head 2024-03-21T12:58:21.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:48887b1c:::145:head 2024-03-21T12:58:21.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:9f24a9dd:::146:head 2024-03-21T12:58:21.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:987f100b:::147:head 2024-03-21T12:58:21.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:d2dd3581:::148:head 2024-03-21T12:58:21.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:7fed1808:::149:head 2024-03-21T12:58:21.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:c80b70e9:::150:head 2024-03-21T12:58:21.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:85ed90f9:::151:head 2024-03-21T12:58:21.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:36428b24:::152:head 2024-03-21T12:58:21.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:d044c34a:::153:head 2024-03-21T12:58:21.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:7c18bf58:::154:head 2024-03-21T12:58:21.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:d1c21232:::155:head 2024-03-21T12:58:21.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:a7a3c575:::156:head 2024-03-21T12:58:21.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:87da0633:::157:head 2024-03-21T12:58:21.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:d5ac3822:::158:head 2024-03-21T12:58:21.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:3f20522d:::159:head 2024-03-21T12:58:21.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:6ca26563:::160:head 2024-03-21T12:58:21.724 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:532ce135:::161:head 2024-03-21T12:58:21.725 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:c78863e6:::162:head 2024-03-21T12:58:21.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:21 smithi043 ceph-mon[41699]: pgmap v412: 356 pgs: 356 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:21.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:21 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_period","val": "600"}]': finished 2024-03-21T12:58:21.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:21 smithi043 ceph-mon[41699]: from='client.36272 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81661-56"}]': finished 2024-03-21T12:58:21.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:21 smithi043 ceph-mon[41699]: osdmap e300: 8 total, 8 up, 8 in 2024-03-21T12:58:21.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:21 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-21T12:58:21.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:21 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-21T12:58:21.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:21 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/483585497' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:21.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:21 smithi043 ceph-mon[41699]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:21.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:21 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/483585497' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:21.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:21 smithi043 ceph-mon[41699]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:21.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:21 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/483585497' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-81661-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:21.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:21 smithi043 ceph-mon[41699]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-81661-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:21.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:21 smithi043 ceph-mon[30062]: pgmap v412: 356 pgs: 356 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:21.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:21 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_period","val": "600"}]': finished 2024-03-21T12:58:21.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:21 smithi043 ceph-mon[30062]: from='client.36272 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81661-56"}]': finished 2024-03-21T12:58:21.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:21 smithi043 ceph-mon[30062]: osdmap e300: 8 total, 8 up, 8 in 2024-03-21T12:58:21.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:21 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-21T12:58:21.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:21 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-21T12:58:21.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:21 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/483585497' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:21.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:21 smithi043 ceph-mon[30062]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:21.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:21 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/483585497' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:21.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:21 smithi043 ceph-mon[30062]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:21.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:21 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/483585497' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-81661-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:21.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:21 smithi043 ceph-mon[30062]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-81661-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:21.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:21 smithi155 ceph-mon[33846]: pgmap v412: 356 pgs: 356 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:21.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:21 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_period","val": "600"}]': finished 2024-03-21T12:58:21.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:21 smithi155 ceph-mon[33846]: from='client.36272 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi043-81661-56"}]': finished 2024-03-21T12:58:21.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:21 smithi155 ceph-mon[33846]: osdmap e300: 8 total, 8 up, 8 in 2024-03-21T12:58:21.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:21 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-21T12:58:21.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:21 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-21T12:58:21.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:21 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/483585497' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:21.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:21 smithi155 ceph-mon[33846]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:21.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:21 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/483585497' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:21.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:21 smithi155 ceph-mon[33846]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:21.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:21 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/483585497' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-81661-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:21.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:21 smithi155 ceph-mon[33846]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-81661-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:23.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:22 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-03-21T12:58:23.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:22 smithi155 ceph-mon[33846]: from='client.36278 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-81661-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:58:23.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:22 smithi155 ceph-mon[33846]: osdmap e301: 8 total, 8 up, 8 in 2024-03-21T12:58:23.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:22 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/483585497' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-81661-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:23.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:22 smithi155 ceph-mon[33846]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-81661-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:23.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:22 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi043-81880-43","var": "pg_num","format": "json"}]: dispatch 2024-03-21T12:58:23.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:22 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:23.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:22 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:23.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:22 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-43"}]: dispatch 2024-03-21T12:58:23.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:22 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-43"}]: dispatch 2024-03-21T12:58:23.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:22 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-43"}]': finished 2024-03-21T12:58:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:22 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-03-21T12:58:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:22 smithi043 ceph-mon[30062]: from='client.36278 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-81661-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:58:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:22 smithi043 ceph-mon[30062]: osdmap e301: 8 total, 8 up, 8 in 2024-03-21T12:58:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:22 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/483585497' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-81661-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:22 smithi043 ceph-mon[30062]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-81661-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:22 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi043-81880-43","var": "pg_num","format": "json"}]: dispatch 2024-03-21T12:58:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:22 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:22 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:22 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-43"}]: dispatch 2024-03-21T12:58:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:22 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-43"}]: dispatch 2024-03-21T12:58:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:22 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-43"}]': finished 2024-03-21T12:58:23.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:22 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-03-21T12:58:23.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:22 smithi043 ceph-mon[41699]: from='client.36278 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi043-81661-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:58:23.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:22 smithi043 ceph-mon[41699]: osdmap e301: 8 total, 8 up, 8 in 2024-03-21T12:58:23.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:22 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/483585497' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-81661-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:23.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:22 smithi043 ceph-mon[41699]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-81661-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:23.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:22 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi043-81880-43","var": "pg_num","format": "json"}]: dispatch 2024-03-21T12:58:23.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:22 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:23.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:22 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:23.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:22 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-43"}]: dispatch 2024-03-21T12:58:23.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:22 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-43"}]: dispatch 2024-03-21T12:58:23.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:22 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-43"}]': finished 2024-03-21T12:58:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:23 smithi155 ceph-mon[33846]: pgmap v415: 356 pgs: 356 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:23 smithi155 ceph-mon[33846]: osdmap e302: 8 total, 8 up, 8 in 2024-03-21T12:58:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:23 smithi155 ceph-mon[33846]: from='client.36278 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-81661-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-81661-57"}]': finished 2024-03-21T12:58:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:23 smithi155 ceph-mon[33846]: osdmap e303: 8 total, 8 up, 8 in 2024-03-21T12:58:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:23 smithi043 ceph-mon[30062]: pgmap v415: 356 pgs: 356 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:23 smithi043 ceph-mon[30062]: osdmap e302: 8 total, 8 up, 8 in 2024-03-21T12:58:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:23 smithi043 ceph-mon[30062]: from='client.36278 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-81661-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-81661-57"}]': finished 2024-03-21T12:58:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:23 smithi043 ceph-mon[30062]: osdmap e303: 8 total, 8 up, 8 in 2024-03-21T12:58:24.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:23 smithi043 ceph-mon[41699]: pgmap v415: 356 pgs: 356 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:24.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:23 smithi043 ceph-mon[41699]: osdmap e302: 8 total, 8 up, 8 in 2024-03-21T12:58:24.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:23 smithi043 ceph-mon[41699]: from='client.36278 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi043-81661-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi043-81661-57"}]': finished 2024-03-21T12:58:24.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:23 smithi043 ceph-mon[41699]: osdmap e303: 8 total, 8 up, 8 in 2024-03-21T12:58:25.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:25 smithi155 ceph-mon[33846]: pgmap v418: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:25.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:25 smithi155 ceph-mon[33846]: osdmap e304: 8 total, 8 up, 8 in 2024-03-21T12:58:25.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:25 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:25.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:25 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:25.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:25 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:58:25.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:25 smithi043 ceph-mon[30062]: pgmap v418: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:25.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:25 smithi043 ceph-mon[30062]: osdmap e304: 8 total, 8 up, 8 in 2024-03-21T12:58:25.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:25 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:25.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:25 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:25.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:25 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:58:25.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:25 smithi043 ceph-mon[41699]: pgmap v418: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:25.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:25 smithi043 ceph-mon[41699]: osdmap e304: 8 total, 8 up, 8 in 2024-03-21T12:58:25.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:25 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:25.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:25 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:25.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:25 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:58:26.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:26 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:58:26.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:26 smithi155 ceph-mon[33846]: osdmap e305: 8 total, 8 up, 8 in 2024-03-21T12:58:26.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:26 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:26.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:26 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/483585497' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:26.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:26 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:26.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:26 smithi155 ceph-mon[33846]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:26.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:26 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:58:26.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:26 smithi043 ceph-mon[30062]: osdmap e305: 8 total, 8 up, 8 in 2024-03-21T12:58:26.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:26 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:26.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:26 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/483585497' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:26.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:26 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:26.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:26 smithi043 ceph-mon[30062]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:26.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:26 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:58:26.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:26 smithi043 ceph-mon[41699]: osdmap e305: 8 total, 8 up, 8 in 2024-03-21T12:58:26.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:26 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:26.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:26 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/483585497' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:26.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:26 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:26.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:26 smithi043 ceph-mon[41699]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:27.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:27 smithi155 ceph-mon[33846]: pgmap v421: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:27.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:27 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-45", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:58:27.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:27 smithi155 ceph-mon[33846]: from='client.36278 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81661-57"}]': finished 2024-03-21T12:58:27.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:27 smithi155 ceph-mon[33846]: osdmap e306: 8 total, 8 up, 8 in 2024-03-21T12:58:27.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:27 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-21T12:58:27.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:27 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/483585497' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:27.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:27 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-21T12:58:27.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:27 smithi155 ceph-mon[33846]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:27.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:27 smithi043 ceph-mon[30062]: pgmap v421: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:27.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:27 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-45", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:58:27.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:27 smithi043 ceph-mon[30062]: from='client.36278 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81661-57"}]': finished 2024-03-21T12:58:27.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:27 smithi043 ceph-mon[30062]: osdmap e306: 8 total, 8 up, 8 in 2024-03-21T12:58:27.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:27 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-21T12:58:27.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:27 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/483585497' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:27.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:27 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-21T12:58:27.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:27 smithi043 ceph-mon[30062]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:27.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:27 smithi043 ceph-mon[41699]: pgmap v421: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:27.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:27 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-45", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:58:27.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:27 smithi043 ceph-mon[41699]: from='client.36278 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi043-81661-57"}]': finished 2024-03-21T12:58:27.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:27 smithi043 ceph-mon[41699]: osdmap e306: 8 total, 8 up, 8 in 2024-03-21T12:58:27.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:27 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-21T12:58:27.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:27 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/483585497' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:27.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:27 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-21T12:58:27.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:27 smithi043 ceph-mon[41699]: from='client.36278 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81661-57"}]: dispatch 2024-03-21T12:58:28.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:58:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:58:28.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:58:28 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:58:28.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:28 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_count","val": "3"}]': finished 2024-03-21T12:58:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:28 smithi155 ceph-mon[33846]: from='client.36278 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81661-57"}]': finished 2024-03-21T12:58:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:28 smithi155 ceph-mon[33846]: osdmap e307: 8 total, 8 up, 8 in 2024-03-21T12:58:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-21T12:58:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:28 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-21T12:58:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-81661-58"}]: dispatch 2024-03-21T12:58:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-81661-58"}]: dispatch 2024-03-21T12:58:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi043-81661-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:28 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_period","val": "3"}]': finished 2024-03-21T12:58:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi043-81661-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:58:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:28 smithi155 ceph-mon[33846]: osdmap e308: 8 total, 8 up, 8 in 2024-03-21T12:58:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T12:58:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi043-81661-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi043-81661-58"}]: dispatch 2024-03-21T12:58:28.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:28 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T12:58:28.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_count","val": "3"}]': finished 2024-03-21T12:58:28.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[30062]: from='client.36278 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81661-57"}]': finished 2024-03-21T12:58:28.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[30062]: osdmap e307: 8 total, 8 up, 8 in 2024-03-21T12:58:28.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-21T12:58:28.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-21T12:58:28.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-81661-58"}]: dispatch 2024-03-21T12:58:28.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-81661-58"}]: dispatch 2024-03-21T12:58:28.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi043-81661-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:28.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_period","val": "3"}]': finished 2024-03-21T12:58:28.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi043-81661-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:58:28.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[30062]: osdmap e308: 8 total, 8 up, 8 in 2024-03-21T12:58:28.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T12:58:28.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi043-81661-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi043-81661-58"}]: dispatch 2024-03-21T12:58:28.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T12:58:28.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_count","val": "3"}]': finished 2024-03-21T12:58:28.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[41699]: from='client.36278 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi043-81661-57"}]': finished 2024-03-21T12:58:28.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[41699]: osdmap e307: 8 total, 8 up, 8 in 2024-03-21T12:58:28.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-21T12:58:28.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-21T12:58:28.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-81661-58"}]: dispatch 2024-03-21T12:58:28.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-81661-58"}]: dispatch 2024-03-21T12:58:28.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi043-81661-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:28.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_period","val": "3"}]': finished 2024-03-21T12:58:28.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi043-81661-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:58:28.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[41699]: osdmap e308: 8 total, 8 up, 8 in 2024-03-21T12:58:28.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T12:58:28.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi043-81661-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi043-81661-58"}]: dispatch 2024-03-21T12:58:28.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:28 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T12:58:30.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:29 smithi155 ceph-mon[33846]: pgmap v424: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:58:30.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:29 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:58:30.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:29 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T12:58:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:29 smithi043 ceph-mon[30062]: pgmap v424: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:58:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:29 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:58:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:29 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T12:58:30.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:29 smithi043 ceph-mon[41699]: pgmap v424: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:58:30.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:29 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:58:30.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:29 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T12:58:31.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:30 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-21T12:58:31.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:30 smithi155 ceph-mon[33846]: osdmap e309: 8 total, 8 up, 8 in 2024-03-21T12:58:31.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:30 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-21T12:58:31.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:30 smithi155 ceph-mon[33846]: pgmap v427: 356 pgs: 356 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:31.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:30 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi043-81661-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi043-81661-58"}]': finished 2024-03-21T12:58:31.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:30 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-21T12:58:31.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:30 smithi155 ceph-mon[33846]: osdmap e310: 8 total, 8 up, 8 in 2024-03-21T12:58:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:30 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-21T12:58:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:30 smithi043 ceph-mon[30062]: osdmap e309: 8 total, 8 up, 8 in 2024-03-21T12:58:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:30 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-21T12:58:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:30 smithi043 ceph-mon[30062]: pgmap v427: 356 pgs: 356 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:30 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi043-81661-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi043-81661-58"}]': finished 2024-03-21T12:58:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:30 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-21T12:58:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:30 smithi043 ceph-mon[30062]: osdmap e310: 8 total, 8 up, 8 in 2024-03-21T12:58:31.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:30 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-21T12:58:31.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:30 smithi043 ceph-mon[41699]: osdmap e309: 8 total, 8 up, 8 in 2024-03-21T12:58:31.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:30 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-21T12:58:31.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:30 smithi043 ceph-mon[41699]: pgmap v427: 356 pgs: 356 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:31.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:30 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi043-81661-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi043-81661-58"}]': finished 2024-03-21T12:58:31.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:30 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-21T12:58:31.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:30 smithi043 ceph-mon[41699]: osdmap e310: 8 total, 8 up, 8 in 2024-03-21T12:58:32.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:32 smithi155 ceph-mon[33846]: osdmap e311: 8 total, 8 up, 8 in 2024-03-21T12:58:32.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:32 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:58:32.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:32 smithi043 ceph-mon[30062]: osdmap e311: 8 total, 8 up, 8 in 2024-03-21T12:58:32.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:32 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:58:32.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:32 smithi043 ceph-mon[41699]: osdmap e311: 8 total, 8 up, 8 in 2024-03-21T12:58:32.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:32 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:58:33.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:33 smithi155 ceph-mon[33846]: pgmap v430: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:33.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:33 smithi155 ceph-mon[33846]: osdmap e312: 8 total, 8 up, 8 in 2024-03-21T12:58:33.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:33 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-81661-58"}]: dispatch 2024-03-21T12:58:33.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:33 smithi043 ceph-mon[30062]: pgmap v430: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:33.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:33 smithi043 ceph-mon[30062]: osdmap e312: 8 total, 8 up, 8 in 2024-03-21T12:58:33.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:33 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-81661-58"}]: dispatch 2024-03-21T12:58:33.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:33 smithi043 ceph-mon[41699]: pgmap v430: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:33.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:33 smithi043 ceph-mon[41699]: osdmap e312: 8 total, 8 up, 8 in 2024-03-21T12:58:33.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:33 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-81661-58"}]: dispatch 2024-03-21T12:58:34.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:34 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-81661-58"}]': finished 2024-03-21T12:58:34.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:34 smithi155 ceph-mon[33846]: osdmap e313: 8 total, 8 up, 8 in 2024-03-21T12:58:34.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:34 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-81661-58"}]: dispatch 2024-03-21T12:58:34.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:34 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:58:34.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:34 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-81661-58"}]': finished 2024-03-21T12:58:34.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:34 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-81661-58"}]': finished 2024-03-21T12:58:34.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:34 smithi043 ceph-mon[41699]: osdmap e313: 8 total, 8 up, 8 in 2024-03-21T12:58:34.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:34 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-81661-58"}]: dispatch 2024-03-21T12:58:34.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:34 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:58:34.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:34 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-81661-58"}]': finished 2024-03-21T12:58:34.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:34 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi043-81661-58"}]': finished 2024-03-21T12:58:34.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:34 smithi043 ceph-mon[30062]: osdmap e313: 8 total, 8 up, 8 in 2024-03-21T12:58:34.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:34 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-81661-58"}]: dispatch 2024-03-21T12:58:34.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:34 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:58:34.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:34 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2088540603' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi043-81661-58"}]': finished 2024-03-21T12:58:35.858 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:35 smithi155 ceph-mon[33846]: pgmap v433: 356 pgs: 356 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-21T12:58:35.859 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:35 smithi155 ceph-mon[33846]: osdmap e314: 8 total, 8 up, 8 in 2024-03-21T12:58:35.859 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:35 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1635676830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:35.859 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:35 smithi155 ceph-mon[33846]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:35.859 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:35 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1635676830' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:35.859 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:35 smithi155 ceph-mon[33846]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:35.859 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:35 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1635676830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-81661-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:35.859 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:35 smithi155 ceph-mon[33846]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-81661-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:35 smithi043 ceph-mon[30062]: pgmap v433: 356 pgs: 356 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-21T12:58:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:35 smithi043 ceph-mon[30062]: osdmap e314: 8 total, 8 up, 8 in 2024-03-21T12:58:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:35 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1635676830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:35 smithi043 ceph-mon[30062]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:35 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1635676830' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:35 smithi043 ceph-mon[30062]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:35 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1635676830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-81661-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:35 smithi043 ceph-mon[30062]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-81661-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:35.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:35 smithi043 ceph-mon[41699]: pgmap v433: 356 pgs: 356 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-21T12:58:35.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:35 smithi043 ceph-mon[41699]: osdmap e314: 8 total, 8 up, 8 in 2024-03-21T12:58:35.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:35 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1635676830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:35.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:35 smithi043 ceph-mon[41699]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:35.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:35 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1635676830' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:35.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:35 smithi043 ceph-mon[41699]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:35.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:35 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1635676830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-81661-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:35.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:35 smithi043 ceph-mon[41699]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-81661-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:36.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:36 smithi155 ceph-mon[33846]: from='client.36290 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-81661-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:58:36.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:36 smithi155 ceph-mon[33846]: osdmap e315: 8 total, 8 up, 8 in 2024-03-21T12:58:36.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1635676830' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-81661-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:36.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:36 smithi155 ceph-mon[33846]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-81661-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:36.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:36 smithi043 ceph-mon[30062]: from='client.36290 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-81661-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:58:36.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:36 smithi043 ceph-mon[30062]: osdmap e315: 8 total, 8 up, 8 in 2024-03-21T12:58:36.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1635676830' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-81661-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:36.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:36 smithi043 ceph-mon[30062]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-81661-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:36.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:36 smithi043 ceph-mon[41699]: from='client.36290 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi043-81661-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:58:36.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:36 smithi043 ceph-mon[41699]: osdmap e315: 8 total, 8 up, 8 in 2024-03-21T12:58:36.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1635676830' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-81661-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:36.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:36 smithi043 ceph-mon[41699]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-81661-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:37.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:37 smithi155 ceph-mon[33846]: pgmap v436: 356 pgs: 356 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-21T12:58:37.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:37 smithi155 ceph-mon[33846]: osdmap e316: 8 total, 8 up, 8 in 2024-03-21T12:58:37.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:37 smithi043 ceph-mon[30062]: pgmap v436: 356 pgs: 356 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-21T12:58:37.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:37 smithi043 ceph-mon[30062]: osdmap e316: 8 total, 8 up, 8 in 2024-03-21T12:58:37.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:37 smithi043 ceph-mon[41699]: pgmap v436: 356 pgs: 356 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-21T12:58:37.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:37 smithi043 ceph-mon[41699]: osdmap e316: 8 total, 8 up, 8 in 2024-03-21T12:58:38.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:58:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:58:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:58:38.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:38 smithi155 ceph-mon[33846]: from='client.36290 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-81661-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-81661-59"}]': finished 2024-03-21T12:58:38.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:38 smithi155 ceph-mon[33846]: osdmap e317: 8 total, 8 up, 8 in 2024-03-21T12:58:38.708 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:58:38 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:58:38.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:38 smithi043 ceph-mon[30062]: from='client.36290 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-81661-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-81661-59"}]': finished 2024-03-21T12:58:38.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:38 smithi043 ceph-mon[30062]: osdmap e317: 8 total, 8 up, 8 in 2024-03-21T12:58:38.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:38 smithi043 ceph-mon[41699]: from='client.36290 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi043-81661-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi043-81661-59"}]': finished 2024-03-21T12:58:38.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:38 smithi043 ceph-mon[41699]: osdmap e317: 8 total, 8 up, 8 in 2024-03-21T12:58:39.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:39 smithi155 ceph-mon[33846]: pgmap v439: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:58:39.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:39 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:58:39.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:39 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:58:39.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:39 smithi155 ceph-mon[33846]: osdmap e318: 8 total, 8 up, 8 in 2024-03-21T12:58:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:39 smithi043 ceph-mon[30062]: pgmap v439: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:58:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:39 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:58:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:39 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:58:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:39 smithi043 ceph-mon[30062]: osdmap e318: 8 total, 8 up, 8 in 2024-03-21T12:58:39.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:39 smithi043 ceph-mon[41699]: pgmap v439: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:58:39.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:39 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:58:39.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:39 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:58:39.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:39 smithi043 ceph-mon[41699]: osdmap e318: 8 total, 8 up, 8 in 2024-03-21T12:58:40.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:40 smithi155 ceph-mon[33846]: osdmap e319: 8 total, 8 up, 8 in 2024-03-21T12:58:40.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:40 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1635676830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:40.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:40 smithi155 ceph-mon[33846]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:40.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:40 smithi043 ceph-mon[30062]: osdmap e319: 8 total, 8 up, 8 in 2024-03-21T12:58:40.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:40 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1635676830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:40.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:40 smithi043 ceph-mon[30062]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:40.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:40 smithi043 ceph-mon[41699]: osdmap e319: 8 total, 8 up, 8 in 2024-03-21T12:58:40.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:40 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1635676830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:40.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:40 smithi043 ceph-mon[41699]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:41.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:7462ddf6:::1o.RoundTripAppendPP (2756 ms) 2024-03-21T12:58:41.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-03-21T12:58:41.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (3263 ms) 2024-03-21T12:58:41.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-03-21T12:58:41.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (2730 ms) 2024-03-21T12:58:41.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-03-21T12:58:41.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3016 ms) 2024-03-21T12:58:41.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-03-21T12:58:41.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: setting pool EIO 2024-03-21T12:58:41.571 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: max_success 99, min_failed 100 2024-03-21T12:58:41.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (4012 ms) 2024-03-21T12:58:41.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-03-21T12:58:41.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3008 ms) 2024-03-21T12:58:41.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (128764 ms total) 2024-03-21T12:58:41.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: 2024-03-21T12:58:41.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-03-21T12:58:41.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-03-21T12:58:41.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (3023 ms) 2024-03-21T12:58:41.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-03-21T12:58:41.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (9032 ms) 2024-03-21T12:58:41.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-03-21T12:58:41.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (14930 ms) 2024-03-21T12:58:41.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-03-21T12:58:41.572 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3260 ms) 2024-03-21T12:58:41.573 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (30245 ms total) 2024-03-21T12:58:41.573 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: 2024-03-21T12:58:41.573 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-03-21T12:58:41.573 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-03-21T12:58:41.573 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (2756 ms) 2024-03-21T12:58:41.573 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (2756 ms total) 2024-03-21T12:58:41.573 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: 2024-03-21T12:58:41.573 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-03-21T12:58:41.573 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-03-21T12:58:41.573 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (13999 ms) 2024-03-21T12:58:41.573 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-03-21T12:58:41.573 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (6974 ms) 2024-03-21T12:58:41.573 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-03-21T12:58:41.573 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (7037 ms) 2024-03-21T12:58:41.573 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-03-21T12:58:41.574 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7017 ms) 2024-03-21T12:58:41.574 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-03-21T12:58:41.574 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3007 ms) 2024-03-21T12:58:41.574 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-03-21T12:58:41.574 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (6944 ms) 2024-03-21T12:58:41.574 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-03-21T12:58:41.574 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (7035 ms) 2024-03-21T12:58:41.574 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-03-21T12:58:41.574 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7018 ms) 2024-03-21T12:58:41.574 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-03-21T12:58:41.574 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7029 ms) 2024-03-21T12:58:41.574 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-03-21T12:58:41.574 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7045 ms) 2024-03-21T12:58:41.574 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-03-21T12:58:41.575 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7027 ms) 2024-03-21T12:58:41.575 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-03-21T12:58:41.575 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7021 ms) 2024-03-21T12:58:41.575 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-03-21T12:58:41.575 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7047 ms) 2024-03-21T12:58:41.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:41 smithi155 ceph-mon[33846]: pgmap v442: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-03-21T12:58:41.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:41 smithi155 ceph-mon[33846]: from='client.36290 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-81661-59"}]': finished 2024-03-21T12:58:41.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:41 smithi155 ceph-mon[33846]: osdmap e320: 8 total, 8 up, 8 in 2024-03-21T12:58:41.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1635676830' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:41.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:41 smithi155 ceph-mon[33846]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:41.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:41 smithi043 ceph-mon[30062]: pgmap v442: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-03-21T12:58:41.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:41 smithi043 ceph-mon[30062]: from='client.36290 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-81661-59"}]': finished 2024-03-21T12:58:41.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:41 smithi043 ceph-mon[30062]: osdmap e320: 8 total, 8 up, 8 in 2024-03-21T12:58:41.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1635676830' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:41.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:41 smithi043 ceph-mon[30062]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:41.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:41 smithi043 ceph-mon[41699]: pgmap v442: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-03-21T12:58:41.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:41 smithi043 ceph-mon[41699]: from='client.36290 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi043-81661-59"}]': finished 2024-03-21T12:58:41.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:41 smithi043 ceph-mon[41699]: osdmap e320: 8 total, 8 up, 8 in 2024-03-21T12:58:41.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1635676830' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:41.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:41 smithi043 ceph-mon[41699]: from='client.36290 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-81661-59"}]: dispatch 2024-03-21T12:58:42.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:42 smithi155 ceph-mon[33846]: from='client.36290 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-81661-59"}]': finished 2024-03-21T12:58:42.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:42 smithi155 ceph-mon[33846]: osdmap e321: 8 total, 8 up, 8 in 2024-03-21T12:58:42.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:42.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:42 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:42.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-45"}]: dispatch 2024-03-21T12:58:42.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:42 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-45"}]: dispatch 2024-03-21T12:58:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:42 smithi043 ceph-mon[30062]: from='client.36290 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-81661-59"}]': finished 2024-03-21T12:58:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:42 smithi043 ceph-mon[30062]: osdmap e321: 8 total, 8 up, 8 in 2024-03-21T12:58:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:42 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-45"}]: dispatch 2024-03-21T12:58:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:42 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-45"}]: dispatch 2024-03-21T12:58:42.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:42 smithi043 ceph-mon[41699]: from='client.36290 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi043-81661-59"}]': finished 2024-03-21T12:58:42.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:42 smithi043 ceph-mon[41699]: osdmap e321: 8 total, 8 up, 8 in 2024-03-21T12:58:42.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:42.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:42 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:42.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-45"}]: dispatch 2024-03-21T12:58:42.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:42 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-45"}]: dispatch 2024-03-21T12:58:43.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:43 smithi155 ceph-mon[33846]: pgmap v445: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-03-21T12:58:43.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:43 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-45"}]': finished 2024-03-21T12:58:43.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:43 smithi155 ceph-mon[33846]: osdmap e322: 8 total, 8 up, 8 in 2024-03-21T12:58:43.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/291268810' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81661-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:43.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:43 smithi155 ceph-mon[33846]: from='client.36296 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81661-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:43.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:43 smithi043 ceph-mon[30062]: pgmap v445: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-03-21T12:58:43.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:43 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-45"}]': finished 2024-03-21T12:58:43.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:43 smithi043 ceph-mon[30062]: osdmap e322: 8 total, 8 up, 8 in 2024-03-21T12:58:43.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/291268810' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81661-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:43.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:43 smithi043 ceph-mon[30062]: from='client.36296 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81661-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:43.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:43 smithi043 ceph-mon[41699]: pgmap v445: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-03-21T12:58:43.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:43 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-45"}]': finished 2024-03-21T12:58:43.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:43 smithi043 ceph-mon[41699]: osdmap e322: 8 total, 8 up, 8 in 2024-03-21T12:58:43.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/291268810' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81661-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:43.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:43 smithi043 ceph-mon[41699]: from='client.36296 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81661-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:44.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:44 smithi155 ceph-mon[33846]: from='client.36296 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81661-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:58:44.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:44 smithi155 ceph-mon[33846]: osdmap e323: 8 total, 8 up, 8 in 2024-03-21T12:58:44.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:44 smithi043 ceph-mon[30062]: from='client.36296 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81661-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:58:44.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:44 smithi043 ceph-mon[30062]: osdmap e323: 8 total, 8 up, 8 in 2024-03-21T12:58:44.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:44 smithi043 ceph-mon[41699]: from='client.36296 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi043-81661-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:58:44.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:44 smithi043 ceph-mon[41699]: osdmap e323: 8 total, 8 up, 8 in 2024-03-21T12:58:45.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:45 smithi155 ceph-mon[33846]: pgmap v448: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:45.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:45 smithi155 ceph-mon[33846]: osdmap e324: 8 total, 8 up, 8 in 2024-03-21T12:58:45.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:45 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:45.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:45 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:45.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:45 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-81661-61"}]: dispatch 2024-03-21T12:58:45.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:45 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-81661-61"}]: dispatch 2024-03-21T12:58:45.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:45 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-81661-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:45.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:45 smithi043 ceph-mon[30062]: pgmap v448: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:45.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:45 smithi043 ceph-mon[30062]: osdmap e324: 8 total, 8 up, 8 in 2024-03-21T12:58:45.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:45.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:45 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:45.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-81661-61"}]: dispatch 2024-03-21T12:58:45.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-81661-61"}]: dispatch 2024-03-21T12:58:45.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-81661-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:45.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:45 smithi043 ceph-mon[41699]: pgmap v448: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:45.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:45 smithi043 ceph-mon[41699]: osdmap e324: 8 total, 8 up, 8 in 2024-03-21T12:58:45.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:45 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:45.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:45 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:45.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:45 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-81661-61"}]: dispatch 2024-03-21T12:58:45.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:45 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-81661-61"}]: dispatch 2024-03-21T12:58:45.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:45 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-81661-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:47.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:46 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:58:47.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-81661-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:58:47.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:46 smithi155 ceph-mon[33846]: osdmap e325: 8 total, 8 up, 8 in 2024-03-21T12:58:47.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-81661-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-81661-61"}]: dispatch 2024-03-21T12:58:47.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:47.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:46 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:47.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:46 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-47", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:58:47.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:46 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:58:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-81661-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:58:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:46 smithi043 ceph-mon[30062]: osdmap e325: 8 total, 8 up, 8 in 2024-03-21T12:58:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-81661-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-81661-61"}]: dispatch 2024-03-21T12:58:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:46 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:46 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-47", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:58:47.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:46 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:58:47.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi043-81661-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:58:47.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:46 smithi043 ceph-mon[41699]: osdmap e325: 8 total, 8 up, 8 in 2024-03-21T12:58:47.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-81661-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-81661-61"}]: dispatch 2024-03-21T12:58:47.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:47.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:46 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:58:47.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:46 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-47", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:58:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:47 smithi155 ceph-mon[33846]: pgmap v451: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-47"}]: dispatch 2024-03-21T12:58:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:47 smithi155 ceph-mon[33846]: osdmap e326: 8 total, 8 up, 8 in 2024-03-21T12:58:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:47 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-47"}]: dispatch 2024-03-21T12:58:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:47 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:58:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-81661-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-81661-61"}]': finished 2024-03-21T12:58:48.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:47 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-47"}]': finished 2024-03-21T12:58:48.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:47 smithi155 ceph-mon[33846]: osdmap e327: 8 total, 8 up, 8 in 2024-03-21T12:58:48.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-47", "mode": "writeback"}]: dispatch 2024-03-21T12:58:48.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:47 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-47", "mode": "writeback"}]: dispatch 2024-03-21T12:58:48.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:58:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:58:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:58:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:47 smithi043 ceph-mon[30062]: pgmap v451: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-47"}]: dispatch 2024-03-21T12:58:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:47 smithi043 ceph-mon[30062]: osdmap e326: 8 total, 8 up, 8 in 2024-03-21T12:58:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:47 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-47"}]: dispatch 2024-03-21T12:58:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:47 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:58:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-81661-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-81661-61"}]': finished 2024-03-21T12:58:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:47 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-47"}]': finished 2024-03-21T12:58:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:47 smithi043 ceph-mon[30062]: osdmap e327: 8 total, 8 up, 8 in 2024-03-21T12:58:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-47", "mode": "writeback"}]: dispatch 2024-03-21T12:58:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:47 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-47", "mode": "writeback"}]: dispatch 2024-03-21T12:58:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:47 smithi043 ceph-mon[41699]: pgmap v451: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-47"}]: dispatch 2024-03-21T12:58:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:47 smithi043 ceph-mon[41699]: osdmap e326: 8 total, 8 up, 8 in 2024-03-21T12:58:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:47 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-47"}]: dispatch 2024-03-21T12:58:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:47 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:58:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi043-81661-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi043-81661-61"}]': finished 2024-03-21T12:58:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:47 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-47"}]': finished 2024-03-21T12:58:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:47 smithi043 ceph-mon[41699]: osdmap e327: 8 total, 8 up, 8 in 2024-03-21T12:58:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-47", "mode": "writeback"}]: dispatch 2024-03-21T12:58:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:47 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-47", "mode": "writeback"}]: dispatch 2024-03-21T12:58:48.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:58:48 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:58:49.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:48 smithi155 ceph-mon[33846]: pgmap v454: 364 pgs: 40 unknown, 324 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:58:49.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:48 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:58:49.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:48 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:58:49.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:48 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-47", "mode": "writeback"}]': finished 2024-03-21T12:58:49.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:48 smithi155 ceph-mon[33846]: osdmap e328: 8 total, 8 up, 8 in 2024-03-21T12:58:49.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:48 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T12:58:49.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:48 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T12:58:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:48 smithi043 ceph-mon[30062]: pgmap v454: 364 pgs: 40 unknown, 324 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:58:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:48 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:58:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:48 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:58:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:48 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-47", "mode": "writeback"}]': finished 2024-03-21T12:58:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:48 smithi043 ceph-mon[30062]: osdmap e328: 8 total, 8 up, 8 in 2024-03-21T12:58:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:48 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T12:58:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:48 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T12:58:49.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:48 smithi043 ceph-mon[41699]: pgmap v454: 364 pgs: 40 unknown, 324 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:58:49.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:48 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:58:49.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:48 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:58:49.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:48 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-47", "mode": "writeback"}]': finished 2024-03-21T12:58:49.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:48 smithi043 ceph-mon[41699]: osdmap e328: 8 total, 8 up, 8 in 2024-03-21T12:58:49.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:48 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T12:58:49.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:48 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T12:58:50.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:50 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:58:50.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:50 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:58:50.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:50 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_count","val": "2"}]': finished 2024-03-21T12:58:50.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:50 smithi155 ceph-mon[33846]: osdmap e329: 8 total, 8 up, 8 in 2024-03-21T12:58:50.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:50 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-81661-61"}]: dispatch 2024-03-21T12:58:50.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:50 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T12:58:50.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:50 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T12:58:50.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:50 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:58:50.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:50 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:58:50.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:50 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_count","val": "2"}]': finished 2024-03-21T12:58:50.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:50 smithi043 ceph-mon[30062]: osdmap e329: 8 total, 8 up, 8 in 2024-03-21T12:58:50.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:50 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-81661-61"}]: dispatch 2024-03-21T12:58:50.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:50 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T12:58:50.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:50 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T12:58:50.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:50 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:58:50.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:50 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:58:50.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:50 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_count","val": "2"}]': finished 2024-03-21T12:58:50.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:50 smithi043 ceph-mon[41699]: osdmap e329: 8 total, 8 up, 8 in 2024-03-21T12:58:50.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:50 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-81661-61"}]: dispatch 2024-03-21T12:58:50.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:50 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T12:58:50.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:50 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T12:58:51.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:51 smithi155 ceph-mon[33846]: pgmap v457: 356 pgs: 356 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-21T12:58:51.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:51 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-81661-61"}]': finished 2024-03-21T12:58:51.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:51 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_period","val": "600"}]': finished 2024-03-21T12:58:51.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:51 smithi155 ceph-mon[33846]: osdmap e330: 8 total, 8 up, 8 in 2024-03-21T12:58:51.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:51 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-81661-61"}]: dispatch 2024-03-21T12:58:51.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:51 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T12:58:51.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:51 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T12:58:51.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:51 smithi043 ceph-mon[30062]: pgmap v457: 356 pgs: 356 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-21T12:58:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:51 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-81661-61"}]': finished 2024-03-21T12:58:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:51 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_period","val": "600"}]': finished 2024-03-21T12:58:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:51 smithi043 ceph-mon[30062]: osdmap e330: 8 total, 8 up, 8 in 2024-03-21T12:58:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:51 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-81661-61"}]: dispatch 2024-03-21T12:58:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:51 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T12:58:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:51 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T12:58:51.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:51 smithi043 ceph-mon[41699]: pgmap v457: 356 pgs: 356 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-21T12:58:51.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:51 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi043-81661-61"}]': finished 2024-03-21T12:58:51.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:51 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_period","val": "600"}]': finished 2024-03-21T12:58:51.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:51 smithi043 ceph-mon[41699]: osdmap e330: 8 total, 8 up, 8 in 2024-03-21T12:58:51.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:51 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-81661-61"}]: dispatch 2024-03-21T12:58:51.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:51 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T12:58:51.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:51 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T12:58:52.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:52 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:58:52.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-81661-61"}]': finished 2024-03-21T12:58:52.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:52 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T12:58:52.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:52 smithi155 ceph-mon[33846]: osdmap e331: 8 total, 8 up, 8 in 2024-03-21T12:58:52.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T12:58:52.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:52 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T12:58:52.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-81661-62"}]: dispatch 2024-03-21T12:58:52.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-81661-62"}]: dispatch 2024-03-21T12:58:52.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-81661-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:52.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:52 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:58:52.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-81661-61"}]': finished 2024-03-21T12:58:52.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:52 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T12:58:52.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:52 smithi043 ceph-mon[30062]: osdmap e331: 8 total, 8 up, 8 in 2024-03-21T12:58:52.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T12:58:52.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:52 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T12:58:52.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-81661-62"}]: dispatch 2024-03-21T12:58:52.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-81661-62"}]: dispatch 2024-03-21T12:58:52.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-81661-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:52.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:52 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:58:52.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/764425488' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi043-81661-61"}]': finished 2024-03-21T12:58:52.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:52 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T12:58:52.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:52 smithi043 ceph-mon[41699]: osdmap e331: 8 total, 8 up, 8 in 2024-03-21T12:58:52.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T12:58:52.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:52 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T12:58:52.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-81661-62"}]: dispatch 2024-03-21T12:58:52.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-81661-62"}]: dispatch 2024-03-21T12:58:52.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-81661-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:53 smithi155 ceph-mon[33846]: pgmap v460: 356 pgs: 356 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-21T12:58:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:53 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-21T12:58:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:53 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-81661-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:58:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:53 smithi155 ceph-mon[33846]: osdmap e332: 8 total, 8 up, 8 in 2024-03-21T12:58:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:53 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-81661-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-81661-62"}]: dispatch 2024-03-21T12:58:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:53 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-21T12:58:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:53 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-21T12:58:54.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:53 smithi043 ceph-mon[30062]: pgmap v460: 356 pgs: 356 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-21T12:58:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:53 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-21T12:58:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:53 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-81661-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:58:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:53 smithi043 ceph-mon[30062]: osdmap e332: 8 total, 8 up, 8 in 2024-03-21T12:58:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:53 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-81661-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-81661-62"}]: dispatch 2024-03-21T12:58:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:53 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-21T12:58:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:53 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-21T12:58:54.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:53 smithi043 ceph-mon[41699]: pgmap v460: 356 pgs: 356 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-21T12:58:54.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:53 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-21T12:58:54.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:53 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi043-81661-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:58:54.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:53 smithi043 ceph-mon[41699]: osdmap e332: 8 total, 8 up, 8 in 2024-03-21T12:58:54.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:53 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-81661-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-81661-62"}]: dispatch 2024-03-21T12:58:54.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:53 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-21T12:58:54.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:53 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-21T12:58:55.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:54 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-21T12:58:55.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:54 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-21T12:58:55.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:54 smithi155 ceph-mon[33846]: osdmap e333: 8 total, 8 up, 8 in 2024-03-21T12:58:55.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:54 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-21T12:58:55.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:54 smithi155 ceph-mon[33846]: pgmap v463: 356 pgs: 356 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:55.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:54 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-81661-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-81661-62"}]': finished 2024-03-21T12:58:55.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:54 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-21T12:58:55.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:54 smithi155 ceph-mon[33846]: osdmap e334: 8 total, 8 up, 8 in 2024-03-21T12:58:55.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:54 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:55.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:54 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:54 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-21T12:58:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:54 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-21T12:58:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:54 smithi043 ceph-mon[30062]: osdmap e333: 8 total, 8 up, 8 in 2024-03-21T12:58:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:54 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-21T12:58:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:54 smithi043 ceph-mon[30062]: pgmap v463: 356 pgs: 356 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:54 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-81661-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-81661-62"}]': finished 2024-03-21T12:58:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:54 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-21T12:58:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:54 smithi043 ceph-mon[30062]: osdmap e334: 8 total, 8 up, 8 in 2024-03-21T12:58:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:54 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:54 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:55.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:54 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-21T12:58:55.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:54 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-21T12:58:55.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:54 smithi043 ceph-mon[41699]: osdmap e333: 8 total, 8 up, 8 in 2024-03-21T12:58:55.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:54 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-21T12:58:55.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:54 smithi043 ceph-mon[41699]: pgmap v463: 356 pgs: 356 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:55.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:54 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi043-81661-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi043-81661-62"}]': finished 2024-03-21T12:58:55.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:54 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-21T12:58:55.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:54 smithi043 ceph-mon[41699]: osdmap e334: 8 total, 8 up, 8 in 2024-03-21T12:58:55.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:54 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:55.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:54 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:56.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:56 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:58:56.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:56 smithi155 ceph-mon[33846]: osdmap e335: 8 total, 8 up, 8 in 2024-03-21T12:58:56.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:56 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-47"}]: dispatch 2024-03-21T12:58:56.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:56 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-47"}]: dispatch 2024-03-21T12:58:56.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:56 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:58:56.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:56 smithi043 ceph-mon[30062]: osdmap e335: 8 total, 8 up, 8 in 2024-03-21T12:58:56.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:56 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-47"}]: dispatch 2024-03-21T12:58:56.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:56 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-47"}]: dispatch 2024-03-21T12:58:56.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:56 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:58:56.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:56 smithi043 ceph-mon[41699]: osdmap e335: 8 total, 8 up, 8 in 2024-03-21T12:58:56.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:56 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-47"}]: dispatch 2024-03-21T12:58:56.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:56 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-47"}]: dispatch 2024-03-21T12:58:57.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:57 smithi155 ceph-mon[33846]: pgmap v466: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:57.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:57 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:58:57.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:57 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:58:57.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:57 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-47"}]': finished 2024-03-21T12:58:57.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:57 smithi155 ceph-mon[33846]: osdmap e336: 8 total, 8 up, 8 in 2024-03-21T12:58:57.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:57 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-81661-62"}]: dispatch 2024-03-21T12:58:57.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:57 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:57.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:57 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:57.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:57 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-47"}]: dispatch 2024-03-21T12:58:57.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:57 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-47"}]: dispatch 2024-03-21T12:58:57.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:57 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:58:57.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:57 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:58:57.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:57 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:58:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[30062]: pgmap v466: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:58:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:58:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-47"}]': finished 2024-03-21T12:58:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[30062]: osdmap e336: 8 total, 8 up, 8 in 2024-03-21T12:58:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-81661-62"}]: dispatch 2024-03-21T12:58:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-47"}]: dispatch 2024-03-21T12:58:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-47"}]: dispatch 2024-03-21T12:58:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:58:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:58:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:58:57.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[41699]: pgmap v466: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:58:57.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:58:57.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:58:57.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-47"}]': finished 2024-03-21T12:58:57.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[41699]: osdmap e336: 8 total, 8 up, 8 in 2024-03-21T12:58:57.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-81661-62"}]: dispatch 2024-03-21T12:58:57.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:57.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:58:57.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-47"}]: dispatch 2024-03-21T12:58:57.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-47"}]: dispatch 2024-03-21T12:58:57.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:58:57.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:58:57.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:57 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:58:58.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:58:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:58:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:58:58.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:58 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-81661-62"}]': finished 2024-03-21T12:58:58.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:58 smithi155 ceph-mon[33846]: osdmap e337: 8 total, 8 up, 8 in 2024-03-21T12:58:58.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:58 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-81661-62"}]: dispatch 2024-03-21T12:58:58.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:58:58 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:58:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:58 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-81661-62"}]': finished 2024-03-21T12:58:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:58 smithi043 ceph-mon[30062]: osdmap e337: 8 total, 8 up, 8 in 2024-03-21T12:58:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:58 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-81661-62"}]: dispatch 2024-03-21T12:58:58.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:58 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi043-81661-62"}]': finished 2024-03-21T12:58:58.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:58 smithi043 ceph-mon[41699]: osdmap e337: 8 total, 8 up, 8 in 2024-03-21T12:58:58.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:58 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-81661-62"}]: dispatch 2024-03-21T12:58:59.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:59 smithi155 ceph-mon[33846]: pgmap v469: 324 pgs: 324 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:58:59.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:59 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:58:59.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:59 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-81661-62"}]': finished 2024-03-21T12:58:59.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:59 smithi155 ceph-mon[33846]: osdmap e338: 8 total, 8 up, 8 in 2024-03-21T12:58:59.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:59 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:59.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:59 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:59.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:59 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2098381672' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:58:59.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:59 smithi155 ceph-mon[33846]: from='client.36314 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:58:59.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:59 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2098381672' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:58:59.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:59 smithi155 ceph-mon[33846]: from='client.36314 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:58:59.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:59 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2098381672' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-81661-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:59.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:58:59 smithi155 ceph-mon[33846]: from='client.36314 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-81661-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:59 smithi043 ceph-mon[30062]: pgmap v469: 324 pgs: 324 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:58:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:59 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:58:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:59 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-81661-62"}]': finished 2024-03-21T12:58:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:59 smithi043 ceph-mon[30062]: osdmap e338: 8 total, 8 up, 8 in 2024-03-21T12:58:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:59 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:59 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:59 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2098381672' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:58:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:59 smithi043 ceph-mon[30062]: from='client.36314 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:58:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:59 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2098381672' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:58:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:59 smithi043 ceph-mon[30062]: from='client.36314 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:58:59.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:59 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2098381672' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-81661-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:59.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:58:59 smithi043 ceph-mon[30062]: from='client.36314 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-81661-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:59.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:59 smithi043 ceph-mon[41699]: pgmap v469: 324 pgs: 324 active+clean; 8.4 MiB data, 637 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:58:59.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:59 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:58:59.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:59 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2091521328' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi043-81661-62"}]': finished 2024-03-21T12:58:59.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:59 smithi043 ceph-mon[41699]: osdmap e338: 8 total, 8 up, 8 in 2024-03-21T12:58:59.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:59 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:59.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:59 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:58:59.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:59 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2098381672' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:58:59.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:59 smithi043 ceph-mon[41699]: from='client.36314 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:58:59.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:59 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2098381672' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:58:59.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:59 smithi043 ceph-mon[41699]: from='client.36314 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:58:59.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:59 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2098381672' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-81661-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:58:59.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:58:59 smithi043 ceph-mon[41699]: from='client.36314 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-81661-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:59:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:00 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:59:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:00 smithi155 ceph-mon[33846]: from='client.36314 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-81661-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:59:01.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:00 smithi155 ceph-mon[33846]: osdmap e339: 8 total, 8 up, 8 in 2024-03-21T12:59:01.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:00 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2098381672' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-81661-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:59:01.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:00 smithi155 ceph-mon[33846]: from='client.36314 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-81661-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:59:01.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:00 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:59:01.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:00 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:59:01.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:00 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-49", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:59:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:00 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:59:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:00 smithi043 ceph-mon[30062]: from='client.36314 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-81661-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:59:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:00 smithi043 ceph-mon[30062]: osdmap e339: 8 total, 8 up, 8 in 2024-03-21T12:59:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:00 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2098381672' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-81661-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:59:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:00 smithi043 ceph-mon[30062]: from='client.36314 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-81661-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:59:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:00 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:59:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:00 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:59:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:00 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-49", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:59:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:00 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:59:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:00 smithi043 ceph-mon[41699]: from='client.36314 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi043-81661-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:59:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:00 smithi043 ceph-mon[41699]: osdmap e339: 8 total, 8 up, 8 in 2024-03-21T12:59:01.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:00 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2098381672' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-81661-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:59:01.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:00 smithi043 ceph-mon[41699]: from='client.36314 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-81661-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:59:01.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:00 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:59:01.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:00 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:59:01.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:00 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-49", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:59:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:01 smithi155 ceph-mon[33846]: pgmap v472: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T12:59:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:01 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-49"}]: dispatch 2024-03-21T12:59:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:01 smithi155 ceph-mon[33846]: osdmap e340: 8 total, 8 up, 8 in 2024-03-21T12:59:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:01 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-49"}]: dispatch 2024-03-21T12:59:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:01 smithi155 ceph-mon[33846]: from='client.36314 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-81661-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-81661-63"}]': finished 2024-03-21T12:59:02.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:01 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-49"}]': finished 2024-03-21T12:59:02.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:01 smithi155 ceph-mon[33846]: osdmap e341: 8 total, 8 up, 8 in 2024-03-21T12:59:02.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:01 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-49", "mode": "readproxy"}]: dispatch 2024-03-21T12:59:02.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:01 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-49", "mode": "readproxy"}]: dispatch 2024-03-21T12:59:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:01 smithi043 ceph-mon[30062]: pgmap v472: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T12:59:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:01 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-49"}]: dispatch 2024-03-21T12:59:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:01 smithi043 ceph-mon[30062]: osdmap e340: 8 total, 8 up, 8 in 2024-03-21T12:59:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:01 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-49"}]: dispatch 2024-03-21T12:59:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:01 smithi043 ceph-mon[30062]: from='client.36314 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-81661-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-81661-63"}]': finished 2024-03-21T12:59:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:01 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-49"}]': finished 2024-03-21T12:59:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:01 smithi043 ceph-mon[30062]: osdmap e341: 8 total, 8 up, 8 in 2024-03-21T12:59:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:01 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-49", "mode": "readproxy"}]: dispatch 2024-03-21T12:59:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:01 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-49", "mode": "readproxy"}]: dispatch 2024-03-21T12:59:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:01 smithi043 ceph-mon[41699]: pgmap v472: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T12:59:02.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:01 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-49"}]: dispatch 2024-03-21T12:59:02.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:01 smithi043 ceph-mon[41699]: osdmap e340: 8 total, 8 up, 8 in 2024-03-21T12:59:02.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:01 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-49"}]: dispatch 2024-03-21T12:59:02.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:01 smithi043 ceph-mon[41699]: from='client.36314 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi043-81661-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi043-81661-63"}]': finished 2024-03-21T12:59:02.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:01 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-49"}]': finished 2024-03-21T12:59:02.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:01 smithi043 ceph-mon[41699]: osdmap e341: 8 total, 8 up, 8 in 2024-03-21T12:59:02.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:01 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-49", "mode": "readproxy"}]: dispatch 2024-03-21T12:59:02.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:01 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-49", "mode": "readproxy"}]: dispatch 2024-03-21T12:59:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:02 smithi155 ceph-mon[33846]: pgmap v475: 364 pgs: 40 unknown, 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T12:59:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:02 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:59:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:02 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:59:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:02 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-49", "mode": "readproxy"}]': finished 2024-03-21T12:59:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:02 smithi155 ceph-mon[33846]: osdmap e342: 8 total, 8 up, 8 in 2024-03-21T12:59:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:02 smithi043 ceph-mon[41699]: pgmap v475: 364 pgs: 40 unknown, 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T12:59:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:02 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:59:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:02 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:59:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:02 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-49", "mode": "readproxy"}]': finished 2024-03-21T12:59:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:02 smithi043 ceph-mon[41699]: osdmap e342: 8 total, 8 up, 8 in 2024-03-21T12:59:03.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:02 smithi043 ceph-mon[30062]: pgmap v475: 364 pgs: 40 unknown, 324 active+clean; 8.4 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T12:59:03.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:02 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:59:03.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:02 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:59:03.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:02 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-49", "mode": "readproxy"}]': finished 2024-03-21T12:59:03.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:02 smithi043 ceph-mon[30062]: osdmap e342: 8 total, 8 up, 8 in 2024-03-21T12:59:04.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:04 smithi155 ceph-mon[33846]: osdmap e343: 8 total, 8 up, 8 in 2024-03-21T12:59:04.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2098381672' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:59:04.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:04 smithi155 ceph-mon[33846]: from='client.36314 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:59:04.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:04 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:59:04.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:04 smithi043 ceph-mon[30062]: osdmap e343: 8 total, 8 up, 8 in 2024-03-21T12:59:04.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2098381672' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:59:04.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:04 smithi043 ceph-mon[30062]: from='client.36314 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:59:04.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:04 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:59:04.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:04 smithi043 ceph-mon[41699]: osdmap e343: 8 total, 8 up, 8 in 2024-03-21T12:59:04.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2098381672' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:59:04.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:04 smithi043 ceph-mon[41699]: from='client.36314 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:59:04.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:04 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:59:05.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:05 smithi155 ceph-mon[33846]: pgmap v478: 356 pgs: 356 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:59:05.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:05 smithi155 ceph-mon[33846]: from='client.36314 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-81661-63"}]': finished 2024-03-21T12:59:05.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:05 smithi155 ceph-mon[33846]: osdmap e344: 8 total, 8 up, 8 in 2024-03-21T12:59:05.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:05 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2098381672' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:59:05.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:05 smithi155 ceph-mon[33846]: from='client.36314 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:59:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:05 smithi043 ceph-mon[30062]: pgmap v478: 356 pgs: 356 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:59:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:05 smithi043 ceph-mon[30062]: from='client.36314 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-81661-63"}]': finished 2024-03-21T12:59:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:05 smithi043 ceph-mon[30062]: osdmap e344: 8 total, 8 up, 8 in 2024-03-21T12:59:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:05 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2098381672' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:59:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:05 smithi043 ceph-mon[30062]: from='client.36314 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:59:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:05 smithi043 ceph-mon[41699]: pgmap v478: 356 pgs: 356 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:59:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:05 smithi043 ceph-mon[41699]: from='client.36314 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi043-81661-63"}]': finished 2024-03-21T12:59:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:05 smithi043 ceph-mon[41699]: osdmap e344: 8 total, 8 up, 8 in 2024-03-21T12:59:05.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:05 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2098381672' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:59:05.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:05 smithi043 ceph-mon[41699]: from='client.36314 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-81661-63"}]: dispatch 2024-03-21T12:59:06.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:06 smithi155 ceph-mon[33846]: from='client.36314 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-81661-63"}]': finished 2024-03-21T12:59:06.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:06 smithi155 ceph-mon[33846]: osdmap e345: 8 total, 8 up, 8 in 2024-03-21T12:59:06.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:06 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2526381023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:06.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:06 smithi155 ceph-mon[33846]: from='client.36320 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:06.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:06 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2526381023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:06.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:06 smithi155 ceph-mon[33846]: from='client.36320 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:06.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:06 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2526381023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-81661-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:59:06.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:06 smithi155 ceph-mon[33846]: from='client.36320 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-81661-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:59:06.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:06 smithi043 ceph-mon[30062]: from='client.36314 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-81661-63"}]': finished 2024-03-21T12:59:06.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:06 smithi043 ceph-mon[30062]: osdmap e345: 8 total, 8 up, 8 in 2024-03-21T12:59:06.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:06 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2526381023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:06.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:06 smithi043 ceph-mon[30062]: from='client.36320 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:06.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:06 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2526381023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:06.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:06 smithi043 ceph-mon[30062]: from='client.36320 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:06.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:06 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2526381023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-81661-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:59:06.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:06 smithi043 ceph-mon[30062]: from='client.36320 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-81661-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:59:06.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:06 smithi043 ceph-mon[41699]: from='client.36314 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi043-81661-63"}]': finished 2024-03-21T12:59:06.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:06 smithi043 ceph-mon[41699]: osdmap e345: 8 total, 8 up, 8 in 2024-03-21T12:59:06.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:06 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2526381023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:06.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:06 smithi043 ceph-mon[41699]: from='client.36320 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:06.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:06 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2526381023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:06.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:06 smithi043 ceph-mon[41699]: from='client.36320 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:06.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:06 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2526381023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-81661-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:59:06.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:06 smithi043 ceph-mon[41699]: from='client.36320 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-81661-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:59:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:07 smithi155 ceph-mon[33846]: pgmap v481: 356 pgs: 356 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:59:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:07 smithi155 ceph-mon[33846]: from='client.36320 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-81661-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:59:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:07 smithi155 ceph-mon[33846]: osdmap e346: 8 total, 8 up, 8 in 2024-03-21T12:59:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:07 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2526381023' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-81661-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:08.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:07 smithi155 ceph-mon[33846]: from='client.36320 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-81661-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:08.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:59:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:59:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:59:08.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:07 smithi043 ceph-mon[30062]: pgmap v481: 356 pgs: 356 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:59:08.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:07 smithi043 ceph-mon[30062]: from='client.36320 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-81661-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:59:08.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:07 smithi043 ceph-mon[30062]: osdmap e346: 8 total, 8 up, 8 in 2024-03-21T12:59:08.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:07 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2526381023' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-81661-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:08.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:07 smithi043 ceph-mon[30062]: from='client.36320 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-81661-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:08.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:07 smithi043 ceph-mon[41699]: pgmap v481: 356 pgs: 356 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T12:59:08.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:07 smithi043 ceph-mon[41699]: from='client.36320 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi043-81661-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:59:08.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:07 smithi043 ceph-mon[41699]: osdmap e346: 8 total, 8 up, 8 in 2024-03-21T12:59:08.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:07 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2526381023' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-81661-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:08.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:07 smithi043 ceph-mon[41699]: from='client.36320 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-81661-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:08.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:59:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:59:09.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:08 smithi155 ceph-mon[33846]: osdmap e347: 8 total, 8 up, 8 in 2024-03-21T12:59:09.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:08 smithi155 ceph-mon[33846]: pgmap v484: 356 pgs: 356 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:59:09.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:08 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:59:09.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:08 smithi155 ceph-mon[33846]: from='client.36320 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-81661-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-81661-64"}]': finished 2024-03-21T12:59:09.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:08 smithi155 ceph-mon[33846]: osdmap e348: 8 total, 8 up, 8 in 2024-03-21T12:59:09.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:08 smithi043 ceph-mon[30062]: osdmap e347: 8 total, 8 up, 8 in 2024-03-21T12:59:09.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:08 smithi043 ceph-mon[30062]: pgmap v484: 356 pgs: 356 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:59:09.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:08 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:59:09.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:08 smithi043 ceph-mon[30062]: from='client.36320 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-81661-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-81661-64"}]': finished 2024-03-21T12:59:09.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:08 smithi043 ceph-mon[30062]: osdmap e348: 8 total, 8 up, 8 in 2024-03-21T12:59:09.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:08 smithi043 ceph-mon[41699]: osdmap e347: 8 total, 8 up, 8 in 2024-03-21T12:59:09.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:08 smithi043 ceph-mon[41699]: pgmap v484: 356 pgs: 356 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:59:09.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:08 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:59:09.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:08 smithi043 ceph-mon[41699]: from='client.36320 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi043-81661-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi043-81661-64"}]': finished 2024-03-21T12:59:09.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:08 smithi043 ceph-mon[41699]: osdmap e348: 8 total, 8 up, 8 in 2024-03-21T12:59:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:10 smithi155 ceph-mon[33846]: osdmap e349: 8 total, 8 up, 8 in 2024-03-21T12:59:10.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:10 smithi043 ceph-mon[30062]: osdmap e349: 8 total, 8 up, 8 in 2024-03-21T12:59:10.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:10 smithi043 ceph-mon[41699]: osdmap e349: 8 total, 8 up, 8 in 2024-03-21T12:59:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:11 smithi155 ceph-mon[33846]: pgmap v487: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:59:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:11 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:59:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:11 smithi155 ceph-mon[33846]: osdmap e350: 8 total, 8 up, 8 in 2024-03-21T12:59:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:11 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2526381023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:11 smithi155 ceph-mon[33846]: from='client.36320 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:11 smithi043 ceph-mon[30062]: pgmap v487: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:59:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:11 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:59:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:11 smithi043 ceph-mon[30062]: osdmap e350: 8 total, 8 up, 8 in 2024-03-21T12:59:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:11 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2526381023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:11 smithi043 ceph-mon[30062]: from='client.36320 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:11.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:11 smithi043 ceph-mon[41699]: pgmap v487: 364 pgs: 8 unknown, 356 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:59:11.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:11 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:59:11.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:11 smithi043 ceph-mon[41699]: osdmap e350: 8 total, 8 up, 8 in 2024-03-21T12:59:11.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:11 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2526381023' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:11.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:11 smithi043 ceph-mon[41699]: from='client.36320 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:12 smithi155 ceph-mon[33846]: from='client.36320 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-81661-64"}]': finished 2024-03-21T12:59:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:12 smithi155 ceph-mon[33846]: osdmap e351: 8 total, 8 up, 8 in 2024-03-21T12:59:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:12 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2526381023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:12 smithi155 ceph-mon[33846]: from='client.36320 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:12 smithi155 ceph-mon[33846]: from='client.36320 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-81661-64"}]': finished 2024-03-21T12:59:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:12 smithi043 ceph-mon[30062]: from='client.36320 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-81661-64"}]': finished 2024-03-21T12:59:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:12 smithi043 ceph-mon[30062]: osdmap e351: 8 total, 8 up, 8 in 2024-03-21T12:59:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:12 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2526381023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:12 smithi043 ceph-mon[30062]: from='client.36320 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:12 smithi043 ceph-mon[30062]: from='client.36320 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-81661-64"}]': finished 2024-03-21T12:59:12.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:12 smithi043 ceph-mon[41699]: from='client.36320 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi043-81661-64"}]': finished 2024-03-21T12:59:12.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:12 smithi043 ceph-mon[41699]: osdmap e351: 8 total, 8 up, 8 in 2024-03-21T12:59:12.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:12 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2526381023' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:12.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:12 smithi043 ceph-mon[41699]: from='client.36320 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-81661-64"}]: dispatch 2024-03-21T12:59:12.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:12 smithi043 ceph-mon[41699]: from='client.36320 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi043-81661-64"}]': finished 2024-03-21T12:59:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:13 smithi155 ceph-mon[33846]: pgmap v490: 356 pgs: 356 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:59:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:13 smithi155 ceph-mon[33846]: osdmap e352: 8 total, 8 up, 8 in 2024-03-21T12:59:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:13 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3269886509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:13 smithi155 ceph-mon[33846]: from='client.36326 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:13 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3269886509' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:13 smithi155 ceph-mon[33846]: from='client.36326 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:13 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3269886509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-81661-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:59:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:13 smithi155 ceph-mon[33846]: from='client.36326 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-81661-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:59:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:13 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:13 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:13 smithi043 ceph-mon[30062]: pgmap v490: 356 pgs: 356 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:59:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:13 smithi043 ceph-mon[30062]: osdmap e352: 8 total, 8 up, 8 in 2024-03-21T12:59:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:13 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3269886509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:13 smithi043 ceph-mon[30062]: from='client.36326 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:13 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3269886509' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:13 smithi043 ceph-mon[30062]: from='client.36326 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:13 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3269886509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-81661-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:59:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:13 smithi043 ceph-mon[30062]: from='client.36326 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-81661-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:59:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:13 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:13 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:13 smithi043 ceph-mon[41699]: pgmap v490: 356 pgs: 356 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:59:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:13 smithi043 ceph-mon[41699]: osdmap e352: 8 total, 8 up, 8 in 2024-03-21T12:59:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:13 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3269886509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:13 smithi043 ceph-mon[41699]: from='client.36326 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:13 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3269886509' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:13 smithi043 ceph-mon[41699]: from='client.36326 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:13 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3269886509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-81661-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:59:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:13 smithi043 ceph-mon[41699]: from='client.36326 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-81661-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:59:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:13 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:13 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:15.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:14 smithi043 ceph-mon[30062]: from='client.36326 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-81661-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:59:15.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:14 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:59:15.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:14 smithi043 ceph-mon[30062]: osdmap e353: 8 total, 8 up, 8 in 2024-03-21T12:59:15.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:14 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-49"}]: dispatch 2024-03-21T12:59:15.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:14 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3269886509' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-81661-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:15.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:14 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-49"}]: dispatch 2024-03-21T12:59:15.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:14 smithi043 ceph-mon[30062]: from='client.36326 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-81661-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:15.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:14 smithi043 ceph-mon[41699]: from='client.36326 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-81661-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:59:15.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:14 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:59:15.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:14 smithi043 ceph-mon[41699]: osdmap e353: 8 total, 8 up, 8 in 2024-03-21T12:59:15.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:14 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-49"}]: dispatch 2024-03-21T12:59:15.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:14 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3269886509' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-81661-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:15.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:14 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-49"}]: dispatch 2024-03-21T12:59:15.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:14 smithi043 ceph-mon[41699]: from='client.36326 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-81661-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:15.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:14 smithi155 ceph-mon[33846]: from='client.36326 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi043-81661-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:59:15.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:14 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:59:15.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:14 smithi155 ceph-mon[33846]: osdmap e353: 8 total, 8 up, 8 in 2024-03-21T12:59:15.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:14 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-49"}]: dispatch 2024-03-21T12:59:15.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:14 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3269886509' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-81661-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:15.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:14 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-49"}]: dispatch 2024-03-21T12:59:15.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:14 smithi155 ceph-mon[33846]: from='client.36326 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-81661-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:15.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP263:head 2024-03-21T12:59:15.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:5d165639:::164:head 2024-03-21T12:59:15.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:f43765fc:::165:head 2024-03-21T12:59:15.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:b4c720e9:::166:head 2024-03-21T12:59:15.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:e694b040:::167:head 2024-03-21T12:59:15.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:afa38db2:::168:head 2024-03-21T12:59:15.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:77ba9f53:::169:head 2024-03-21T12:59:15.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:87495034:::170:head 2024-03-21T12:59:15.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:7c96bf0e:::171:head 2024-03-21T12:59:15.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:dbe346cc:::172:head 2024-03-21T12:59:15.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:e943ec24:::173:head 2024-03-21T12:59:15.708 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:f97a9c0c:::174:head 2024-03-21T12:59:15.708 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:6f26e74d:::175:head 2024-03-21T12:59:15.708 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:4f95e106:::176:head 2024-03-21T12:59:15.708 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:0e6f2f8f:::177:head 2024-03-21T12:59:15.708 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:05db05f1:::178:head 2024-03-21T12:59:15.708 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:38a78d66:::179:head 2024-03-21T12:59:15.708 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:d095610b:::180:head 2024-03-21T12:59:15.708 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:a1a9d709:::181:head 2024-03-21T12:59:15.708 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:1e5d39db:::182:head 2024-03-21T12:59:15.708 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:f7df4fb9:::183:head 2024-03-21T12:59:15.708 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:03a7f161:::184:head 2024-03-21T12:59:15.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:ba70721e:::185:head 2024-03-21T12:59:15.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:28e5662d:::186:head 2024-03-21T12:59:15.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:973d52de:::187:head 2024-03-21T12:59:15.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:4303eb1c:::188:head 2024-03-21T12:59:15.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:b990b48e:::189:head 2024-03-21T12:59:15.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:29b8165b:::190:head 2024-03-21T12:59:15.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:3547f197:::191:head 2024-03-21T12:59:15.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:7e260936:::192:head 2024-03-21T12:59:15.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:1abec7b1:::193:head 2024-03-21T12:59:15.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:10fdda93:::194:head 2024-03-21T12:59:15.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:15817eea:::195:head 2024-03-21T12:59:15.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:770bab57:::196:head 2024-03-21T12:59:15.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:ed9e13e7:::197:head 2024-03-21T12:59:15.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:71471a8f:::198:head 2024-03-21T12:59:15.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: checking for 265:10fb1d02:::199:head 2024-03-21T12:59:15.710 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (7784 ms) 2024-03-21T12:59:15.710 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-03-21T12:59:15.710 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711025911,0 2024-03-21T12:59:15.710 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: first is 1711025911 2024-03-21T12:59:15.710 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711025911,0 2024-03-21T12:59:15.710 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711025911,0 2024-03-21T12:59:15.710 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711025911,1711025913,1711025914,0 2024-03-21T12:59:15.710 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711025911,1711025913,1711025914,0 2024-03-21T12:59:15.710 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711025911,1711025913,1711025914,0 2024-03-21T12:59:15.710 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711025911,1711025913,1711025914,1711025916,1711025917,0 2024-03-21T12:59:15.711 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711025911,1711025913,1711025914,1711025916,1711025917,0 2024-03-21T12:59:15.711 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711025911,1711025913,1711025914,1711025916,1711025917,0 2024-03-21T12:59:15.711 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711025911,1711025913,1711025914,1711025916,1711025917,1711025919,1711025920,0 2024-03-21T12:59:15.711 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711025911,1711025913,1711025914,1711025916,1711025917,1711025919,1711025920,0 2024-03-21T12:59:15.711 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711025911,1711025913,1711025914,1711025916,1711025917,1711025919,1711025920,0 2024-03-21T12:59:15.711 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711025914,1711025916,1711025917,1711025919,1711025920,1711025922,1711025923,0 2024-03-21T12:59:15.711 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: first now 1711025914, trimmed 2024-03-21T12:59:15.711 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (20100 ms) 2024-03-21T12:59:15.711 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-03-21T12:59:15.711 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: foo0 2024-03-21T12:59:15.711 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-03-21T12:59:15.711 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14056 ms) 2024-03-21T12:59:15.711 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-03-21T12:59:16.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:15 smithi043 ceph-mon[30062]: pgmap v493: 356 pgs: 356 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:59:16.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:15 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:59:16.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:15 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-49"}]': finished 2024-03-21T12:59:16.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:15 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:16.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:15 smithi043 ceph-mon[30062]: osdmap e354: 8 total, 8 up, 8 in 2024-03-21T12:59:16.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:15 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:16.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:15 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-49"}]: dispatch 2024-03-21T12:59:16.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:15 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-49"}]: dispatch 2024-03-21T12:59:16.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:15 smithi043 ceph-mon[30062]: from='client.36326 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-81661-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-81661-65"}]': finished 2024-03-21T12:59:16.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:15 smithi043 ceph-mon[30062]: osdmap e355: 8 total, 8 up, 8 in 2024-03-21T12:59:16.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:15 smithi043 ceph-mon[41699]: pgmap v493: 356 pgs: 356 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:59:16.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:15 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:59:16.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:15 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-49"}]': finished 2024-03-21T12:59:16.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:15 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:16.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:15 smithi043 ceph-mon[41699]: osdmap e354: 8 total, 8 up, 8 in 2024-03-21T12:59:16.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:15 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:16.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:15 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-49"}]: dispatch 2024-03-21T12:59:16.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:15 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-49"}]: dispatch 2024-03-21T12:59:16.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:15 smithi043 ceph-mon[41699]: from='client.36326 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-81661-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-81661-65"}]': finished 2024-03-21T12:59:16.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:15 smithi043 ceph-mon[41699]: osdmap e355: 8 total, 8 up, 8 in 2024-03-21T12:59:16.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:15 smithi155 ceph-mon[33846]: pgmap v493: 356 pgs: 356 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:59:16.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:15 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:59:16.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:15 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-49"}]': finished 2024-03-21T12:59:16.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:15 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:16.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:15 smithi155 ceph-mon[33846]: osdmap e354: 8 total, 8 up, 8 in 2024-03-21T12:59:16.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:15 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:16.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:15 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-49"}]: dispatch 2024-03-21T12:59:16.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:15 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-49"}]: dispatch 2024-03-21T12:59:16.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:15 smithi155 ceph-mon[33846]: from='client.36326 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi043-81661-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi043-81661-65"}]': finished 2024-03-21T12:59:16.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:15 smithi155 ceph-mon[33846]: osdmap e355: 8 total, 8 up, 8 in 2024-03-21T12:59:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:17 smithi043 ceph-mon[30062]: pgmap v496: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:59:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:17 smithi043 ceph-mon[30062]: osdmap e356: 8 total, 8 up, 8 in 2024-03-21T12:59:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:17 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:17 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:17 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:59:17.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:17 smithi043 ceph-mon[41699]: pgmap v496: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:59:17.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:17 smithi043 ceph-mon[41699]: osdmap e356: 8 total, 8 up, 8 in 2024-03-21T12:59:17.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:17 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:17.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:17 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:17.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:17 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:59:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:17 smithi155 ceph-mon[33846]: pgmap v496: 332 pgs: 8 unknown, 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T12:59:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:17 smithi155 ceph-mon[33846]: osdmap e356: 8 total, 8 up, 8 in 2024-03-21T12:59:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:17 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:17 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:18.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:17 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:59:18.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:59:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:59:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:59:18.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:59:18 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:59:18.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:18 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:59:18.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:18 smithi043 ceph-mon[30062]: osdmap e357: 8 total, 8 up, 8 in 2024-03-21T12:59:18.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:18 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3269886509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:18.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:18 smithi043 ceph-mon[30062]: from='client.36326 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:18.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:18 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:59:18.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:18 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:59:18.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:18 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:59:18.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:18 smithi043 ceph-mon[41699]: osdmap e357: 8 total, 8 up, 8 in 2024-03-21T12:59:18.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:18 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3269886509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:18.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:18 smithi043 ceph-mon[41699]: from='client.36326 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:18.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:18 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:59:18.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:18 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:59:19.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:18 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:59:19.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:18 smithi155 ceph-mon[33846]: osdmap e357: 8 total, 8 up, 8 in 2024-03-21T12:59:19.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:18 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3269886509' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:19.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:18 smithi155 ceph-mon[33846]: from='client.36326 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:19.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:18 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:59:19.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:18 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T12:59:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:19 smithi043 ceph-mon[30062]: pgmap v499: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:59:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:19 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:59:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:19 smithi043 ceph-mon[30062]: from='client.36326 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81661-65"}]': finished 2024-03-21T12:59:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:19 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-51", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:59:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:19 smithi043 ceph-mon[30062]: osdmap e358: 8 total, 8 up, 8 in 2024-03-21T12:59:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:19 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-51"}]: dispatch 2024-03-21T12:59:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:19 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3269886509' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:19 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-51"}]: dispatch 2024-03-21T12:59:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:19 smithi043 ceph-mon[30062]: from='client.36326 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:19.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:19 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:59:19.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:19 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:59:19.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:19 smithi043 ceph-mon[41699]: pgmap v499: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:59:19.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:19 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:59:19.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:19 smithi043 ceph-mon[41699]: from='client.36326 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81661-65"}]': finished 2024-03-21T12:59:19.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:19 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-51", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:59:19.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:19 smithi043 ceph-mon[41699]: osdmap e358: 8 total, 8 up, 8 in 2024-03-21T12:59:19.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:19 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-51"}]: dispatch 2024-03-21T12:59:19.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:19 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3269886509' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:19.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:19 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-51"}]: dispatch 2024-03-21T12:59:19.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:19 smithi043 ceph-mon[41699]: from='client.36326 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:19.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:19 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:59:19.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:19 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:59:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:19 smithi155 ceph-mon[33846]: pgmap v499: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:59:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:19 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:59:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:19 smithi155 ceph-mon[33846]: from='client.36326 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi043-81661-65"}]': finished 2024-03-21T12:59:20.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:19 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-51", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T12:59:20.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:19 smithi155 ceph-mon[33846]: osdmap e358: 8 total, 8 up, 8 in 2024-03-21T12:59:20.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:19 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-51"}]: dispatch 2024-03-21T12:59:20.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:19 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3269886509' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:20.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:19 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-51"}]: dispatch 2024-03-21T12:59:20.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:19 smithi155 ceph-mon[33846]: from='client.36326 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81661-65"}]: dispatch 2024-03-21T12:59:20.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:19 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:59:20.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:19 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:59:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:20 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-51"}]': finished 2024-03-21T12:59:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:20 smithi155 ceph-mon[33846]: from='client.36326 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81661-65"}]': finished 2024-03-21T12:59:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:20 smithi155 ceph-mon[33846]: osdmap e359: 8 total, 8 up, 8 in 2024-03-21T12:59:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:20 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-51", "mode": "writeback"}]: dispatch 2024-03-21T12:59:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:20 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-51", "mode": "writeback"}]: dispatch 2024-03-21T12:59:21.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:20 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1916613907' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:21.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:20 smithi155 ceph-mon[33846]: from='client.36332 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:21.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:20 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1916613907' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:21.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:20 smithi155 ceph-mon[33846]: from='client.36332 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:21.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:20 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1916613907' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-81661-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:59:21.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:20 smithi155 ceph-mon[33846]: from='client.36332 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-81661-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:59:21.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:20 smithi155 ceph-mon[33846]: pgmap v502: 356 pgs: 356 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-21T12:59:21.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:20 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:59:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-51"}]': finished 2024-03-21T12:59:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[30062]: from='client.36326 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81661-65"}]': finished 2024-03-21T12:59:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[30062]: osdmap e359: 8 total, 8 up, 8 in 2024-03-21T12:59:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-51", "mode": "writeback"}]: dispatch 2024-03-21T12:59:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-51", "mode": "writeback"}]: dispatch 2024-03-21T12:59:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1916613907' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[30062]: from='client.36332 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1916613907' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[30062]: from='client.36332 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1916613907' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-81661-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:59:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[30062]: from='client.36332 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-81661-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:59:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[30062]: pgmap v502: 356 pgs: 356 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-21T12:59:21.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:59:21.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-51"}]': finished 2024-03-21T12:59:21.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[41699]: from='client.36326 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi043-81661-65"}]': finished 2024-03-21T12:59:21.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[41699]: osdmap e359: 8 total, 8 up, 8 in 2024-03-21T12:59:21.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-51", "mode": "writeback"}]: dispatch 2024-03-21T12:59:21.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-51", "mode": "writeback"}]: dispatch 2024-03-21T12:59:21.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1916613907' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:21.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[41699]: from='client.36332 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:21.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1916613907' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:21.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[41699]: from='client.36332 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:21.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1916613907' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-81661-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:59:21.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[41699]: from='client.36332 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-81661-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T12:59:21.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[41699]: pgmap v502: 356 pgs: 356 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-21T12:59:21.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:20 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T12:59:22.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:21 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-51", "mode": "writeback"}]': finished 2024-03-21T12:59:22.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:21 smithi155 ceph-mon[33846]: from='client.36332 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-81661-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:59:22.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:21 smithi155 ceph-mon[33846]: osdmap e360: 8 total, 8 up, 8 in 2024-03-21T12:59:22.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:21 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1916613907' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-81661-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:22.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:21 smithi155 ceph-mon[33846]: from='client.36332 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-81661-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:22.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:21 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T12:59:22.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:21 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T12:59:22.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:21 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_count","val": "2"}]': finished 2024-03-21T12:59:22.483 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:21 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-51", "mode": "writeback"}]': finished 2024-03-21T12:59:22.483 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:21 smithi043 ceph-mon[41699]: from='client.36332 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-81661-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:59:22.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:21 smithi043 ceph-mon[41699]: osdmap e360: 8 total, 8 up, 8 in 2024-03-21T12:59:22.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:21 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1916613907' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-81661-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:22.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:21 smithi043 ceph-mon[41699]: from='client.36332 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-81661-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:22.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:21 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T12:59:22.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:21 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T12:59:22.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:21 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_count","val": "2"}]': finished 2024-03-21T12:59:22.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:21 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-51", "mode": "writeback"}]': finished 2024-03-21T12:59:22.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:21 smithi043 ceph-mon[30062]: from='client.36332 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi043-81661-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T12:59:22.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:21 smithi043 ceph-mon[30062]: osdmap e360: 8 total, 8 up, 8 in 2024-03-21T12:59:22.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:21 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1916613907' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-81661-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:22.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:21 smithi043 ceph-mon[30062]: from='client.36332 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-81661-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:22.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:21 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T12:59:22.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:21 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T12:59:22.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:21 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_count","val": "2"}]': finished 2024-03-21T12:59:23.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:22 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T12:59:23.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:22 smithi155 ceph-mon[33846]: osdmap e361: 8 total, 8 up, 8 in 2024-03-21T12:59:23.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:22 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T12:59:23.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:22 smithi155 ceph-mon[33846]: pgmap v505: 356 pgs: 356 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-21T12:59:23.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:22 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:59:23.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:22 smithi155 ceph-mon[33846]: from='client.36332 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-81661-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-81661-66"}]': finished 2024-03-21T12:59:23.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:22 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_period","val": "600"}]': finished 2024-03-21T12:59:23.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:22 smithi155 ceph-mon[33846]: osdmap e362: 8 total, 8 up, 8 in 2024-03-21T12:59:23.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:22 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T12:59:23.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:22 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T12:59:23.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:22 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T12:59:23.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:22 smithi043 ceph-mon[30062]: osdmap e361: 8 total, 8 up, 8 in 2024-03-21T12:59:23.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:22 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T12:59:23.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:22 smithi043 ceph-mon[30062]: pgmap v505: 356 pgs: 356 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-21T12:59:23.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:22 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:59:23.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:22 smithi043 ceph-mon[30062]: from='client.36332 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-81661-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-81661-66"}]': finished 2024-03-21T12:59:23.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:22 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_period","val": "600"}]': finished 2024-03-21T12:59:23.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:22 smithi043 ceph-mon[30062]: osdmap e362: 8 total, 8 up, 8 in 2024-03-21T12:59:23.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:22 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T12:59:23.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:22 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T12:59:23.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:22 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T12:59:23.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:22 smithi043 ceph-mon[41699]: osdmap e361: 8 total, 8 up, 8 in 2024-03-21T12:59:23.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:22 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T12:59:23.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:22 smithi043 ceph-mon[41699]: pgmap v505: 356 pgs: 356 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-21T12:59:23.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:22 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:59:23.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:22 smithi043 ceph-mon[41699]: from='client.36332 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi043-81661-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi043-81661-66"}]': finished 2024-03-21T12:59:23.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:22 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_period","val": "600"}]': finished 2024-03-21T12:59:23.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:22 smithi043 ceph-mon[41699]: osdmap e362: 8 total, 8 up, 8 in 2024-03-21T12:59:23.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:22 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T12:59:23.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:22 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T12:59:24.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:23 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:59:24.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:23 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T12:59:24.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:23 smithi155 ceph-mon[33846]: osdmap e363: 8 total, 8 up, 8 in 2024-03-21T12:59:24.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:23 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T12:59:24.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:23 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T12:59:24.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:23 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:59:24.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:23 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T12:59:24.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:23 smithi043 ceph-mon[30062]: osdmap e363: 8 total, 8 up, 8 in 2024-03-21T12:59:24.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:23 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T12:59:24.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:23 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T12:59:24.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:23 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T12:59:24.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:23 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T12:59:24.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:23 smithi043 ceph-mon[41699]: osdmap e363: 8 total, 8 up, 8 in 2024-03-21T12:59:24.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:23 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T12:59:24.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:23 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T12:59:25.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:24 smithi155 ceph-mon[33846]: pgmap v508: 364 pgs: 7 creating+peering, 1 unknown, 356 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-21T12:59:25.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:24 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-21T12:59:25.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:24 smithi155 ceph-mon[33846]: osdmap e364: 8 total, 8 up, 8 in 2024-03-21T12:59:25.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:24 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-21T12:59:25.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:24 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1916613907' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:25.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:24 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-21T12:59:25.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:24 smithi155 ceph-mon[33846]: from='client.36332 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:25.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:24 smithi043 ceph-mon[30062]: pgmap v508: 364 pgs: 7 creating+peering, 1 unknown, 356 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-21T12:59:25.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:24 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-21T12:59:25.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:24 smithi043 ceph-mon[30062]: osdmap e364: 8 total, 8 up, 8 in 2024-03-21T12:59:25.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:24 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-21T12:59:25.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:24 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1916613907' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:25.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:24 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-21T12:59:25.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:24 smithi043 ceph-mon[30062]: from='client.36332 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:25.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:24 smithi043 ceph-mon[41699]: pgmap v508: 364 pgs: 7 creating+peering, 1 unknown, 356 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-21T12:59:25.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:24 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-21T12:59:25.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:24 smithi043 ceph-mon[41699]: osdmap e364: 8 total, 8 up, 8 in 2024-03-21T12:59:25.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:24 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-21T12:59:25.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:24 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1916613907' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:25.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:24 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-21T12:59:25.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:24 smithi043 ceph-mon[41699]: from='client.36332 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:26.750 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ 2024-03-21T12:59:26.750 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3027 ms) 2024-03-21T12:59:26.750 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-03-21T12:59:26.750 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7017 ms) 2024-03-21T12:59:26.750 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-03-21T12:59:26.751 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7032 ms) 2024-03-21T12:59:26.751 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-03-21T12:59:26.751 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7014 ms) 2024-03-21T12:59:26.751 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-03-21T12:59:26.751 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7030 ms) 2024-03-21T12:59:26.751 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-03-21T12:59:26.751 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7039 ms) 2024-03-21T12:59:26.751 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-03-21T12:59:26.751 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7017 ms) 2024-03-21T12:59:26.751 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139376 ms total) 2024-03-21T12:59:26.751 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: 2024-03-21T12:59:26.751 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-03-21T12:59:26.752 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (301141 ms total) 2024-03-21T12:59:26.752 INFO:tasks.workunit.client.0.smithi043.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-03-21T12:59:27.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:26 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "target_max_objects","val": "1"}]': finished 2024-03-21T12:59:27.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:26 smithi155 ceph-mon[33846]: from='client.36332 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-81661-66"}]': finished 2024-03-21T12:59:27.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:26 smithi155 ceph-mon[33846]: osdmap e365: 8 total, 8 up, 8 in 2024-03-21T12:59:27.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:26 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/1916613907' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:27.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:26 smithi155 ceph-mon[33846]: from='client.36332 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:27.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:26 smithi155 ceph-mon[33846]: pgmap v511: 356 pgs: 356 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-21T12:59:27.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:26 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "target_max_objects","val": "1"}]': finished 2024-03-21T12:59:27.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:26 smithi043 ceph-mon[41699]: from='client.36332 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-81661-66"}]': finished 2024-03-21T12:59:27.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:26 smithi043 ceph-mon[41699]: osdmap e365: 8 total, 8 up, 8 in 2024-03-21T12:59:27.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:26 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/1916613907' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:27.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:26 smithi043 ceph-mon[41699]: from='client.36332 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:27.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:26 smithi043 ceph-mon[41699]: pgmap v511: 356 pgs: 356 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-21T12:59:27.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:26 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-51","var": "target_max_objects","val": "1"}]': finished 2024-03-21T12:59:27.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:26 smithi043 ceph-mon[30062]: from='client.36332 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi043-81661-66"}]': finished 2024-03-21T12:59:27.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:26 smithi043 ceph-mon[30062]: osdmap e365: 8 total, 8 up, 8 in 2024-03-21T12:59:27.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:26 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/1916613907' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:27.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:26 smithi043 ceph-mon[30062]: from='client.36332 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-81661-66"}]: dispatch 2024-03-21T12:59:27.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:26 smithi043 ceph-mon[30062]: pgmap v511: 356 pgs: 356 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-21T12:59:28.023 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:27 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-21T12:59:28.023 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:27 smithi043 ceph-mon[30062]: from='client.36332 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-81661-66"}]': finished 2024-03-21T12:59:28.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:27 smithi043 ceph-mon[30062]: osdmap e366: 8 total, 8 up, 8 in 2024-03-21T12:59:28.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:27 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-21T12:59:28.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:27 smithi043 ceph-mon[41699]: from='client.36332 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-81661-66"}]': finished 2024-03-21T12:59:28.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:27 smithi043 ceph-mon[41699]: osdmap e366: 8 total, 8 up, 8 in 2024-03-21T12:59:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:27 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-21T12:59:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:27 smithi155 ceph-mon[33846]: from='client.36332 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi043-81661-66"}]': finished 2024-03-21T12:59:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:27 smithi155 ceph-mon[33846]: osdmap e366: 8 total, 8 up, 8 in 2024-03-21T12:59:28.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:59:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:59:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:59:28.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:59:28 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:59:29.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:28 smithi155 ceph-mon[33846]: pgmap v513: 356 pgs: 356 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:59:29.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:28 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:59:29.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:28 smithi043 ceph-mon[30062]: pgmap v513: 356 pgs: 356 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:59:29.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:28 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:59:29.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:28 smithi043 ceph-mon[41699]: pgmap v513: 356 pgs: 356 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:59:29.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:28 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:59:31.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:31 smithi155 ceph-mon[33846]: pgmap v514: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T12:59:31.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:31 smithi043 ceph-mon[30062]: pgmap v514: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T12:59:31.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:31 smithi043 ceph-mon[41699]: pgmap v514: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T12:59:33.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:33 smithi155 ceph-mon[33846]: pgmap v515: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 704 B/s rd, 0 B/s wr, 0 op/s 2024-03-21T12:59:33.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:33 smithi043 ceph-mon[30062]: pgmap v515: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 704 B/s rd, 0 B/s wr, 0 op/s 2024-03-21T12:59:33.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:33 smithi043 ceph-mon[41699]: pgmap v515: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 704 B/s rd, 0 B/s wr, 0 op/s 2024-03-21T12:59:34.582 INFO:tasks.workunit.client.0.smithi043.stdout: started 0x55855252c850 2024-03-21T12:59:34.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252c9a0 2024-03-21T12:59:34.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252ce50 2024-03-21T12:59:34.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252cf80 2024-03-21T12:59:34.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252d0b0 2024-03-21T12:59:34.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252d430 2024-03-21T12:59:34.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252d8e0 2024-03-21T12:59:34.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252da10 2024-03-21T12:59:34.582 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252c120 2024-03-21T12:59:34.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252c250 2024-03-21T12:59:34.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252c380 2024-03-21T12:59:34.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252db40 2024-03-21T12:59:34.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252e030 2024-03-21T12:59:34.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252e160 2024-03-21T12:59:34.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252d540 2024-03-21T12:59:34.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252d670 2024-03-21T12:59:34.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252d7a0 2024-03-21T12:59:34.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252e2b0 2024-03-21T12:59:34.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252e760 2024-03-21T12:59:34.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252e8d0 2024-03-21T12:59:34.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252dc70 2024-03-21T12:59:34.583 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252dd80 2024-03-21T12:59:34.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252ded0 2024-03-21T12:59:34.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252ea20 2024-03-21T12:59:34.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252eef0 2024-03-21T12:59:34.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252f020 2024-03-21T12:59:34.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252e3c0 2024-03-21T12:59:34.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252e510 2024-03-21T12:59:34.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252e620 2024-03-21T12:59:34.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252f150 2024-03-21T12:59:34.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252f600 2024-03-21T12:59:34.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252f730 2024-03-21T12:59:34.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252fd30 2024-03-21T12:59:34.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252f260 2024-03-21T12:59:34.584 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252f390 2024-03-21T12:59:34.585 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252f4c0 2024-03-21T12:59:34.585 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252fe60 2024-03-21T12:59:34.585 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252f880 2024-03-21T12:59:34.585 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252eb50 2024-03-21T12:59:34.585 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252eca0 2024-03-21T12:59:34.585 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252edd0 2024-03-21T12:59:34.585 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552530460 2024-03-21T12:59:34.585 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252ffb0 2024-03-21T12:59:34.585 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525300e0 2024-03-21T12:59:34.585 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552530230 2024-03-21T12:59:34.585 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552530570 2024-03-21T12:59:34.585 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552530a40 2024-03-21T12:59:34.585 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552530b70 2024-03-21T12:59:34.586 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252f990 2024-03-21T12:59:34.586 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252fac0 2024-03-21T12:59:34.586 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855252fbf0 2024-03-21T12:59:34.586 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552530ca0 2024-03-21T12:59:34.586 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552531150 2024-03-21T12:59:34.586 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525312a0 2024-03-21T12:59:34.586 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552530680 2024-03-21T12:59:34.586 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525307d0 2024-03-21T12:59:34.586 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552530900 2024-03-21T12:59:34.586 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525313f0 2024-03-21T12:59:34.586 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525318c0 2024-03-21T12:59:34.586 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552531a10 2024-03-21T12:59:34.587 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552531ff0 2024-03-21T12:59:34.587 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552531520 2024-03-21T12:59:34.587 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552531650 2024-03-21T12:59:34.587 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552531780 2024-03-21T12:59:34.587 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552532140 2024-03-21T12:59:34.587 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552531b40 2024-03-21T12:59:34.587 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552530db0 2024-03-21T12:59:34.587 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552530f00 2024-03-21T12:59:34.587 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552532270 2024-03-21T12:59:34.587 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552532740 2024-03-21T12:59:34.587 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552532870 2024-03-21T12:59:34.587 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552532e30 2024-03-21T12:59:34.588 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525323a0 2024-03-21T12:59:34.588 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552531030 2024-03-21T12:59:34.588 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525324b0 2024-03-21T12:59:34.588 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525325e0 2024-03-21T12:59:34.588 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552532980 2024-03-21T12:59:34.588 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552532ab0 2024-03-21T12:59:34.588 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552532c00 2024-03-21T12:59:34.588 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552532f80 2024-03-21T12:59:34.588 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552533470 2024-03-21T12:59:34.588 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525335a0 2024-03-21T12:59:35.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:35 smithi155 ceph-mon[33846]: pgmap v516: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T12:59:35.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:35 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:59:35.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:59:35.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:35 smithi043 ceph-mon[30062]: pgmap v516: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T12:59:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:35 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:59:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:59:35.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:35 smithi043 ceph-mon[41699]: pgmap v516: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T12:59:35.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:35 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:59:35.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:59:36.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:36.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:36 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:36.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:36 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:36.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:36.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:36 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:37.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:37 smithi155 ceph-mon[33846]: pgmap v517: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T12:59:37.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:37 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:59:37.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:37 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-51"}]: dispatch 2024-03-21T12:59:37.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:37 smithi155 ceph-mon[33846]: osdmap e367: 8 total, 8 up, 8 in 2024-03-21T12:59:37.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:37 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-51"}]: dispatch 2024-03-21T12:59:37.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:37 smithi043 ceph-mon[30062]: pgmap v517: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T12:59:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:37 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:59:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:37 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-51"}]: dispatch 2024-03-21T12:59:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:37 smithi043 ceph-mon[30062]: osdmap e367: 8 total, 8 up, 8 in 2024-03-21T12:59:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:37 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-51"}]: dispatch 2024-03-21T12:59:37.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:37 smithi043 ceph-mon[41699]: pgmap v517: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T12:59:37.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:37 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T12:59:37.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:37 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-51"}]: dispatch 2024-03-21T12:59:37.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:37 smithi043 ceph-mon[41699]: osdmap e367: 8 total, 8 up, 8 in 2024-03-21T12:59:37.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:37 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-51"}]: dispatch 2024-03-21T12:59:38.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:59:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:59:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:59:38.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:38 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-51"}]': finished 2024-03-21T12:59:38.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:38 smithi043 ceph-mon[41699]: osdmap e368: 8 total, 8 up, 8 in 2024-03-21T12:59:38.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:38 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:38.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:38 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:38.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:38 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-51"}]: dispatch 2024-03-21T12:59:38.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:38 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-51"}]: dispatch 2024-03-21T12:59:38.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:38 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-51"}]': finished 2024-03-21T12:59:38.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:38 smithi043 ceph-mon[30062]: osdmap e368: 8 total, 8 up, 8 in 2024-03-21T12:59:38.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:38 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:38.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:38 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:38.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:38 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-51"}]: dispatch 2024-03-21T12:59:38.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:38 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-51"}]: dispatch 2024-03-21T12:59:38.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:59:38 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:59:38.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:38 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-51"}]': finished 2024-03-21T12:59:38.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:38 smithi155 ceph-mon[33846]: osdmap e368: 8 total, 8 up, 8 in 2024-03-21T12:59:38.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:38 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:38.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:38 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:38.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:38 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-51"}]: dispatch 2024-03-21T12:59:38.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:38 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-51"}]: dispatch 2024-03-21T12:59:39.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:39 smithi155 ceph-mon[33846]: pgmap v520: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T12:59:39.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:39 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-21T12:59:39.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:39 smithi155 ceph-mon[33846]: osdmap e369: 8 total, 8 up, 8 in 2024-03-21T12:59:39.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:39 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:59:39.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:39 smithi043 ceph-mon[30062]: pgmap v520: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T12:59:39.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:39 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-21T12:59:39.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:39 smithi043 ceph-mon[30062]: osdmap e369: 8 total, 8 up, 8 in 2024-03-21T12:59:39.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:39 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:59:39.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:39 smithi043 ceph-mon[41699]: pgmap v520: 356 pgs: 356 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T12:59:39.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:39 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-21T12:59:39.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:39 smithi043 ceph-mon[41699]: osdmap e369: 8 total, 8 up, 8 in 2024-03-21T12:59:39.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:39 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:59:40.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:40 smithi155 ceph-mon[33846]: osdmap e370: 8 total, 8 up, 8 in 2024-03-21T12:59:40.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:40 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:40.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:40 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:40.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:40 smithi043 ceph-mon[30062]: osdmap e370: 8 total, 8 up, 8 in 2024-03-21T12:59:40.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:40 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:40.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:40 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:40.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:40 smithi043 ceph-mon[41699]: osdmap e370: 8 total, 8 up, 8 in 2024-03-21T12:59:40.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:40 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:40.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:40 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:41.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:41 smithi155 ceph-mon[33846]: pgmap v523: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-21T12:59:41.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:41 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:59:41.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:41 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:59:41.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:41 smithi155 ceph-mon[33846]: osdmap e371: 8 total, 8 up, 8 in 2024-03-21T12:59:41.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:41.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:41 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:41.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-53"}]: dispatch 2024-03-21T12:59:41.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:41 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-53"}]: dispatch 2024-03-21T12:59:41.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:41 smithi043 ceph-mon[30062]: pgmap v523: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-21T12:59:41.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:41 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:59:41.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:41 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:59:41.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:41 smithi043 ceph-mon[30062]: osdmap e371: 8 total, 8 up, 8 in 2024-03-21T12:59:41.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:41.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:41 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:41.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-53"}]: dispatch 2024-03-21T12:59:41.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:41 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-53"}]: dispatch 2024-03-21T12:59:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:41 smithi043 ceph-mon[41699]: pgmap v523: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-21T12:59:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:41 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:59:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:41 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:59:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:41 smithi043 ceph-mon[41699]: osdmap e371: 8 total, 8 up, 8 in 2024-03-21T12:59:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:41.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:41 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:41.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-53"}]: dispatch 2024-03-21T12:59:41.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:41 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-53"}]: dispatch 2024-03-21T12:59:42.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:42 smithi155 ceph-mon[33846]: osdmap e372: 8 total, 8 up, 8 in 2024-03-21T12:59:42.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:42 smithi043 ceph-mon[30062]: osdmap e372: 8 total, 8 up, 8 in 2024-03-21T12:59:42.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:42 smithi043 ceph-mon[41699]: osdmap e372: 8 total, 8 up, 8 in 2024-03-21T12:59:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:43 smithi155 ceph-mon[33846]: pgmap v526: 324 pgs: 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-21T12:59:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:43 smithi155 ceph-mon[33846]: osdmap e373: 8 total, 8 up, 8 in 2024-03-21T12:59:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:43 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:43 smithi043 ceph-mon[30062]: pgmap v526: 324 pgs: 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-21T12:59:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:43 smithi043 ceph-mon[30062]: osdmap e373: 8 total, 8 up, 8 in 2024-03-21T12:59:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:43 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:43.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:43 smithi043 ceph-mon[41699]: pgmap v526: 324 pgs: 324 active+clean; 8.4 MiB data, 642 MiB used, 715 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-21T12:59:43.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:43 smithi043 ceph-mon[41699]: osdmap e373: 8 total, 8 up, 8 in 2024-03-21T12:59:43.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:43.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:43 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:44.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:44 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:59:44.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:44 smithi155 ceph-mon[33846]: osdmap e374: 8 total, 8 up, 8 in 2024-03-21T12:59:44.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:44 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T12:59:44.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:44 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:44.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:44 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:44.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:44 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-55"}]: dispatch 2024-03-21T12:59:44.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:44 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-55"}]: dispatch 2024-03-21T12:59:44.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:44 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:59:44.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:44 smithi043 ceph-mon[30062]: osdmap e374: 8 total, 8 up, 8 in 2024-03-21T12:59:44.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:44 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T12:59:44.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:44 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:44.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:44 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:44.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:44 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-55"}]: dispatch 2024-03-21T12:59:44.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:44 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-55"}]: dispatch 2024-03-21T12:59:44.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:44 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:59:44.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:44 smithi043 ceph-mon[41699]: osdmap e374: 8 total, 8 up, 8 in 2024-03-21T12:59:44.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:44 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T12:59:44.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:44 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:44.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:44 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:44.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:44 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-55"}]: dispatch 2024-03-21T12:59:44.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:44 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-55"}]: dispatch 2024-03-21T12:59:45.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:45 smithi155 ceph-mon[33846]: pgmap v529: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T12:59:45.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:45 smithi155 ceph-mon[33846]: osdmap e375: 8 total, 8 up, 8 in 2024-03-21T12:59:45.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:45 smithi043 ceph-mon[30062]: pgmap v529: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T12:59:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:45 smithi043 ceph-mon[30062]: osdmap e375: 8 total, 8 up, 8 in 2024-03-21T12:59:45.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:45 smithi043 ceph-mon[41699]: pgmap v529: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T12:59:45.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:45 smithi043 ceph-mon[41699]: osdmap e375: 8 total, 8 up, 8 in 2024-03-21T12:59:46.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:46 smithi155 ceph-mon[33846]: osdmap e376: 8 total, 8 up, 8 in 2024-03-21T12:59:46.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:46.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:46 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:46.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:46 smithi043 ceph-mon[30062]: osdmap e376: 8 total, 8 up, 8 in 2024-03-21T12:59:46.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:46.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:46 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:46.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:46 smithi043 ceph-mon[41699]: osdmap e376: 8 total, 8 up, 8 in 2024-03-21T12:59:46.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:46.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:46 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:47.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:47 smithi155 ceph-mon[33846]: pgmap v532: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T12:59:47.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:47 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:59:47.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:47 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:59:47.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:47 smithi155 ceph-mon[33846]: osdmap e377: 8 total, 8 up, 8 in 2024-03-21T12:59:47.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:47.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:47 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:47.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-57"}]: dispatch 2024-03-21T12:59:47.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:47 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-57"}]: dispatch 2024-03-21T12:59:47.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:47 smithi043 ceph-mon[30062]: pgmap v532: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T12:59:47.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:47 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:59:47.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:47 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:59:47.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:47 smithi043 ceph-mon[30062]: osdmap e377: 8 total, 8 up, 8 in 2024-03-21T12:59:47.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:47.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:47 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:47.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-57"}]: dispatch 2024-03-21T12:59:47.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:47 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-57"}]: dispatch 2024-03-21T12:59:47.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:47 smithi043 ceph-mon[41699]: pgmap v532: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T12:59:47.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:47 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:59:47.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:47 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:59:47.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:47 smithi043 ceph-mon[41699]: osdmap e377: 8 total, 8 up, 8 in 2024-03-21T12:59:47.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:47.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:47 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:47.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-57"}]: dispatch 2024-03-21T12:59:47.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:47 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-57"}]: dispatch 2024-03-21T12:59:48.322 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:59:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:59:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:59:48.709 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:48 smithi155 ceph-mon[33846]: osdmap e378: 8 total, 8 up, 8 in 2024-03-21T12:59:48.710 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:59:48 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:59:48.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:48 smithi043 ceph-mon[30062]: osdmap e378: 8 total, 8 up, 8 in 2024-03-21T12:59:48.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:48 smithi043 ceph-mon[41699]: osdmap e378: 8 total, 8 up, 8 in 2024-03-21T12:59:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:49 smithi155 ceph-mon[33846]: pgmap v535: 324 pgs: 324 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:59:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:49 smithi155 ceph-mon[33846]: osdmap e379: 8 total, 8 up, 8 in 2024-03-21T12:59:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:49 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:49 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:49 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:59:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:49 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:59:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:49 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:59:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:49 smithi043 ceph-mon[30062]: pgmap v535: 324 pgs: 324 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:59:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:49 smithi043 ceph-mon[30062]: osdmap e379: 8 total, 8 up, 8 in 2024-03-21T12:59:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:49 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:49 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:49 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:59:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:49 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:59:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:49 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:59:49.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:49 smithi043 ceph-mon[41699]: pgmap v535: 324 pgs: 324 active+clean; 8.4 MiB data, 647 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:59:49.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:49 smithi043 ceph-mon[41699]: osdmap e379: 8 total, 8 up, 8 in 2024-03-21T12:59:49.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:49 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:49 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:49 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:59:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:49 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:59:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:49 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T12:59:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:50 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:59:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:50 smithi155 ceph-mon[33846]: osdmap e380: 8 total, 8 up, 8 in 2024-03-21T12:59:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:50 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T12:59:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:50 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:50 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:50 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-59"}]: dispatch 2024-03-21T12:59:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:50 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-59"}]: dispatch 2024-03-21T12:59:50.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:50 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:59:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:50 smithi043 ceph-mon[30062]: osdmap e380: 8 total, 8 up, 8 in 2024-03-21T12:59:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:50 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T12:59:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:50 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:50 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:50 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-59"}]: dispatch 2024-03-21T12:59:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:50 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-59"}]: dispatch 2024-03-21T12:59:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:50 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:59:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:50 smithi043 ceph-mon[41699]: osdmap e380: 8 total, 8 up, 8 in 2024-03-21T12:59:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:50 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T12:59:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:50 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:50 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:50 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-59"}]: dispatch 2024-03-21T12:59:50.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:50 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-59"}]: dispatch 2024-03-21T12:59:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:51 smithi155 ceph-mon[33846]: pgmap v538: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T12:59:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:51 smithi155 ceph-mon[33846]: osdmap e381: 8 total, 8 up, 8 in 2024-03-21T12:59:51.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:51 smithi043 ceph-mon[30062]: pgmap v538: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T12:59:51.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:51 smithi043 ceph-mon[30062]: osdmap e381: 8 total, 8 up, 8 in 2024-03-21T12:59:51.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:51 smithi043 ceph-mon[41699]: pgmap v538: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T12:59:51.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:51 smithi043 ceph-mon[41699]: osdmap e381: 8 total, 8 up, 8 in 2024-03-21T12:59:52.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:52 smithi155 ceph-mon[33846]: osdmap e382: 8 total, 8 up, 8 in 2024-03-21T12:59:52.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:52.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:52 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:52.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:52 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:59:52.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:52 smithi155 ceph-mon[33846]: osdmap e383: 8 total, 8 up, 8 in 2024-03-21T12:59:52.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:52 smithi043 ceph-mon[41699]: osdmap e382: 8 total, 8 up, 8 in 2024-03-21T12:59:52.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:52.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:52 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:52.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:52 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:59:52.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:52 smithi043 ceph-mon[41699]: osdmap e383: 8 total, 8 up, 8 in 2024-03-21T12:59:52.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:52 smithi043 ceph-mon[30062]: osdmap e382: 8 total, 8 up, 8 in 2024-03-21T12:59:52.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:52.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:52 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:52.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:52 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:59:52.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:52 smithi043 ceph-mon[30062]: osdmap e383: 8 total, 8 up, 8 in 2024-03-21T12:59:53.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:53 smithi155 ceph-mon[33846]: pgmap v541: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T12:59:53.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:53 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T12:59:53.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:53 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:59:53.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:53 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T12:59:53.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:53 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T12:59:53.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:53 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T12:59:53.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:53 smithi155 ceph-mon[33846]: osdmap e384: 8 total, 8 up, 8 in 2024-03-21T12:59:53.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:53 smithi043 ceph-mon[30062]: pgmap v541: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T12:59:53.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:53 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T12:59:53.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:53 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:59:53.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:53 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T12:59:53.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:53 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T12:59:53.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:53 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T12:59:53.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:53 smithi043 ceph-mon[30062]: osdmap e384: 8 total, 8 up, 8 in 2024-03-21T12:59:53.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:53 smithi043 ceph-mon[41699]: pgmap v541: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T12:59:53.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:53 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T12:59:53.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:53 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:59:53.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:53 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T12:59:53.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:53 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T12:59:53.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:53 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T12:59:53.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:53 smithi043 ceph-mon[41699]: osdmap e384: 8 total, 8 up, 8 in 2024-03-21T12:59:54.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:54 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:54.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:54 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:54.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:54 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-61"}]: dispatch 2024-03-21T12:59:54.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:54 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-61"}]: dispatch 2024-03-21T12:59:54.733 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:54 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:54.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:54 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:54.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:54 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-61"}]: dispatch 2024-03-21T12:59:54.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:54 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-61"}]: dispatch 2024-03-21T12:59:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:54 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:54 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T12:59:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:54 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-61"}]: dispatch 2024-03-21T12:59:54.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:54 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-61"}]: dispatch 2024-03-21T12:59:55.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:55 smithi155 ceph-mon[33846]: pgmap v544: 356 pgs: 356 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T12:59:55.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:55 smithi155 ceph-mon[33846]: osdmap e385: 8 total, 8 up, 8 in 2024-03-21T12:59:55.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:55 smithi043 ceph-mon[30062]: pgmap v544: 356 pgs: 356 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T12:59:55.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:55 smithi043 ceph-mon[30062]: osdmap e385: 8 total, 8 up, 8 in 2024-03-21T12:59:55.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:55 smithi043 ceph-mon[41699]: pgmap v544: 356 pgs: 356 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T12:59:55.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:55 smithi043 ceph-mon[41699]: osdmap e385: 8 total, 8 up, 8 in 2024-03-21T12:59:56.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:56 smithi155 ceph-mon[33846]: osdmap e386: 8 total, 8 up, 8 in 2024-03-21T12:59:56.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:56 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:56.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:56 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:56.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:56 smithi043 ceph-mon[30062]: osdmap e386: 8 total, 8 up, 8 in 2024-03-21T12:59:56.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:56 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:56.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:56 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:56.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:56 smithi043 ceph-mon[41699]: osdmap e386: 8 total, 8 up, 8 in 2024-03-21T12:59:56.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:56 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:56.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:56 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T12:59:57.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:57 smithi155 ceph-mon[33846]: pgmap v547: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T12:59:57.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:57 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:59:57.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:57 smithi155 ceph-mon[33846]: osdmap e387: 8 total, 8 up, 8 in 2024-03-21T12:59:57.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:57 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T12:59:57.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:57 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T12:59:57.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:57 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T12:59:57.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:57 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:59:57.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:57 smithi043 ceph-mon[30062]: pgmap v547: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T12:59:57.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:57 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:59:57.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:57 smithi043 ceph-mon[30062]: osdmap e387: 8 total, 8 up, 8 in 2024-03-21T12:59:57.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:57 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T12:59:57.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:57 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T12:59:57.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:57 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T12:59:57.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:57 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:59:57.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:57 smithi043 ceph-mon[41699]: pgmap v547: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T12:59:57.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:57 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T12:59:57.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:57 smithi043 ceph-mon[41699]: osdmap e387: 8 total, 8 up, 8 in 2024-03-21T12:59:57.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:57 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T12:59:57.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:57 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T12:59:57.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:57 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T12:59:57.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:57 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T12:59:58.404 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 12:59:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:12:59:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T12:59:58.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 12:59:58 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T12:59:58.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:58 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T12:59:58.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:58 smithi155 ceph-mon[33846]: osdmap e388: 8 total, 8 up, 8 in 2024-03-21T12:59:58.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:58 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:59:58.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:58 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:59:58.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:58 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:59:58.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:58 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T12:59:58.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:58 smithi043 ceph-mon[30062]: osdmap e388: 8 total, 8 up, 8 in 2024-03-21T12:59:58.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:58 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:59:58.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:58 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:59:58.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:58 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:59:58.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:58 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T12:59:58.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:58 smithi043 ceph-mon[41699]: osdmap e388: 8 total, 8 up, 8 in 2024-03-21T12:59:58.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:58 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T12:59:58.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:58 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T12:59:58.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:58 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T12:59:59.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:59 smithi155 ceph-mon[33846]: pgmap v550: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:59:59.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:59 smithi155 ceph-mon[33846]: osdmap e389: 8 total, 8 up, 8 in 2024-03-21T12:59:59.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:59 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:59:59.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 12:59:59 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:59:59.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:59 smithi043 ceph-mon[30062]: pgmap v550: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:59:59.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:59 smithi043 ceph-mon[30062]: osdmap e389: 8 total, 8 up, 8 in 2024-03-21T12:59:59.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:59 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:59:59.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 12:59:59 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T12:59:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:59 smithi043 ceph-mon[41699]: pgmap v550: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-03-21T12:59:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:59 smithi043 ceph-mon[41699]: osdmap e389: 8 total, 8 up, 8 in 2024-03-21T12:59:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:59 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T12:59:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 12:59:59 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:00:00.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:00 smithi155 ceph-mon[33846]: osdmap e390: 8 total, 8 up, 8 in 2024-03-21T13:00:00.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:00 smithi155 ceph-mon[33846]: Health detail: HEALTH_WARN 5 pool(s) do not have an application enabled 2024-03-21T13:00:00.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:00 smithi155 ceph-mon[33846]: [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled 2024-03-21T13:00:00.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:00 smithi155 ceph-mon[33846]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-21T13:00:00.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:00 smithi155 ceph-mon[33846]: application not enabled on pool 'SnapListsmithi043-82997-1' 2024-03-21T13:00:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:00 smithi155 ceph-mon[33846]: application not enabled on pool 'WatchNotifysmithi043-83023-1' 2024-03-21T13:00:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:00 smithi155 ceph-mon[33846]: application not enabled on pool 'poolrace.1804289383' 2024-03-21T13:00:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:00 smithi155 ceph-mon[33846]: application not enabled on pool 'poolrace2.846930886' 2024-03-21T13:00:00.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:00 smithi155 ceph-mon[33846]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-21T13:00:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:00 smithi043 ceph-mon[30062]: osdmap e390: 8 total, 8 up, 8 in 2024-03-21T13:00:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:00 smithi043 ceph-mon[30062]: Health detail: HEALTH_WARN 5 pool(s) do not have an application enabled 2024-03-21T13:00:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:00 smithi043 ceph-mon[30062]: [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled 2024-03-21T13:00:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:00 smithi043 ceph-mon[30062]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-21T13:00:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:00 smithi043 ceph-mon[30062]: application not enabled on pool 'SnapListsmithi043-82997-1' 2024-03-21T13:00:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:00 smithi043 ceph-mon[30062]: application not enabled on pool 'WatchNotifysmithi043-83023-1' 2024-03-21T13:00:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:00 smithi043 ceph-mon[30062]: application not enabled on pool 'poolrace.1804289383' 2024-03-21T13:00:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:00 smithi043 ceph-mon[30062]: application not enabled on pool 'poolrace2.846930886' 2024-03-21T13:00:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:00 smithi043 ceph-mon[30062]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-21T13:00:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:00 smithi043 ceph-mon[41699]: osdmap e390: 8 total, 8 up, 8 in 2024-03-21T13:00:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:00 smithi043 ceph-mon[41699]: Health detail: HEALTH_WARN 5 pool(s) do not have an application enabled 2024-03-21T13:00:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:00 smithi043 ceph-mon[41699]: [WRN] POOL_APP_NOT_ENABLED: 5 pool(s) do not have an application enabled 2024-03-21T13:00:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:00 smithi043 ceph-mon[41699]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-21T13:00:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:00 smithi043 ceph-mon[41699]: application not enabled on pool 'SnapListsmithi043-82997-1' 2024-03-21T13:00:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:00 smithi043 ceph-mon[41699]: application not enabled on pool 'WatchNotifysmithi043-83023-1' 2024-03-21T13:00:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:00 smithi043 ceph-mon[41699]: application not enabled on pool 'poolrace.1804289383' 2024-03-21T13:00:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:00 smithi043 ceph-mon[41699]: application not enabled on pool 'poolrace2.846930886' 2024-03-21T13:00:00.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:00 smithi043 ceph-mon[41699]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-21T13:00:01.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:01 smithi155 ceph-mon[33846]: pgmap v553: 356 pgs: 356 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-21T13:00:01.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:01 smithi155 ceph-mon[33846]: osdmap e391: 8 total, 8 up, 8 in 2024-03-21T13:00:01.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:01 smithi043 ceph-mon[30062]: pgmap v553: 356 pgs: 356 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-21T13:00:01.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:01 smithi043 ceph-mon[30062]: osdmap e391: 8 total, 8 up, 8 in 2024-03-21T13:00:01.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:01 smithi043 ceph-mon[41699]: pgmap v553: 356 pgs: 356 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-21T13:00:01.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:01 smithi043 ceph-mon[41699]: osdmap e391: 8 total, 8 up, 8 in 2024-03-21T13:00:02.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:02 smithi155 ceph-mon[33846]: osdmap e392: 8 total, 8 up, 8 in 2024-03-21T13:00:02.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:02 smithi043 ceph-mon[30062]: osdmap e392: 8 total, 8 up, 8 in 2024-03-21T13:00:02.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:02 smithi043 ceph-mon[41699]: osdmap e392: 8 total, 8 up, 8 in 2024-03-21T13:00:03.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:03 smithi155 ceph-mon[33846]: pgmap v556: 356 pgs: 356 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-21T13:00:03.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:03 smithi043 ceph-mon[30062]: pgmap v556: 356 pgs: 356 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-21T13:00:03.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:03 smithi043 ceph-mon[41699]: pgmap v556: 356 pgs: 356 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-21T13:00:04.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:04 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:00:04.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:04 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:00:04.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:04 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:00:05.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:05 smithi155 ceph-mon[33846]: pgmap v557: 356 pgs: 356 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-03-21T13:00:05.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:05 smithi043 ceph-mon[30062]: pgmap v557: 356 pgs: 356 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-03-21T13:00:05.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:05 smithi043 ceph-mon[41699]: pgmap v557: 356 pgs: 356 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-03-21T13:00:07.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:07 smithi155 ceph-mon[33846]: pgmap v558: 356 pgs: 356 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 929 B/s wr, 3 op/s 2024-03-21T13:00:07.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:07 smithi043 ceph-mon[30062]: pgmap v558: 356 pgs: 356 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 929 B/s wr, 3 op/s 2024-03-21T13:00:07.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:07 smithi043 ceph-mon[41699]: pgmap v558: 356 pgs: 356 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 929 B/s wr, 3 op/s 2024-03-21T13:00:08.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:00:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:00:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:00:08.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:08 smithi155 ceph-mon[33846]: osdmap e393: 8 total, 8 up, 8 in 2024-03-21T13:00:08.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:00:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:00:08.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:08 smithi043 ceph-mon[30062]: osdmap e393: 8 total, 8 up, 8 in 2024-03-21T13:00:08.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:08 smithi043 ceph-mon[41699]: osdmap e393: 8 total, 8 up, 8 in 2024-03-21T13:00:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:09 smithi155 ceph-mon[33846]: pgmap v560: 356 pgs: 356 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 808 B/s wr, 2 op/s 2024-03-21T13:00:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:09 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:00:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:09 smithi043 ceph-mon[30062]: pgmap v560: 356 pgs: 356 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 808 B/s wr, 2 op/s 2024-03-21T13:00:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:09 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:00:09.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:09 smithi043 ceph-mon[41699]: pgmap v560: 356 pgs: 356 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 808 B/s wr, 2 op/s 2024-03-21T13:00:09.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:09 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:00:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:11 smithi155 ceph-mon[33846]: pgmap v561: 356 pgs: 356 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 715 B/s wr, 3 op/s 2024-03-21T13:00:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:11 smithi043 ceph-mon[30062]: pgmap v561: 356 pgs: 356 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 715 B/s wr, 3 op/s 2024-03-21T13:00:11.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:11 smithi043 ceph-mon[41699]: pgmap v561: 356 pgs: 356 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 715 B/s wr, 3 op/s 2024-03-21T13:00:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:12 smithi155 ceph-mon[33846]: osdmap e394: 8 total, 8 up, 8 in 2024-03-21T13:00:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:12 smithi043 ceph-mon[30062]: osdmap e394: 8 total, 8 up, 8 in 2024-03-21T13:00:12.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:12 smithi043 ceph-mon[41699]: osdmap e394: 8 total, 8 up, 8 in 2024-03-21T13:00:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:13 smithi155 ceph-mon[33846]: pgmap v563: 356 pgs: 356 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:00:13.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:13 smithi043 ceph-mon[30062]: pgmap v563: 356 pgs: 356 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:00:13.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:13 smithi043 ceph-mon[41699]: pgmap v563: 356 pgs: 356 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:00:15.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:15 smithi155 ceph-mon[33846]: pgmap v564: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:00:15.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:15 smithi043 ceph-mon[30062]: pgmap v564: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:00:15.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:15 smithi043 ceph-mon[41699]: pgmap v564: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:00:17.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:17 smithi155 ceph-mon[33846]: pgmap v565: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:00:17.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:17 smithi155 ceph-mon[33846]: osdmap e395: 8 total, 8 up, 8 in 2024-03-21T13:00:17.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:17 smithi043 ceph-mon[30062]: pgmap v565: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:00:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:17 smithi043 ceph-mon[30062]: osdmap e395: 8 total, 8 up, 8 in 2024-03-21T13:00:17.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:17 smithi043 ceph-mon[41699]: pgmap v565: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:00:17.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:17 smithi043 ceph-mon[41699]: osdmap e395: 8 total, 8 up, 8 in 2024-03-21T13:00:18.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:00:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:00:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:00:18.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:00:18 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:00:19.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:19 smithi155 ceph-mon[33846]: pgmap v567: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:00:19.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:19 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:00:19.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:19 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:00:19.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:19 smithi043 ceph-mon[30062]: pgmap v567: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:00:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:19 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:00:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:19 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:00:19.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:19 smithi043 ceph-mon[41699]: pgmap v567: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:00:19.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:19 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:00:19.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:19 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:00:21.962 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:21 smithi155 ceph-mon[33846]: pgmap v568: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:00:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:21 smithi043 ceph-mon[30062]: pgmap v568: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:00:21.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:21 smithi043 ceph-mon[41699]: pgmap v568: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:00:22.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:22 smithi155 ceph-mon[33846]: osdmap e396: 8 total, 8 up, 8 in 2024-03-21T13:00:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:22 smithi043 ceph-mon[30062]: osdmap e396: 8 total, 8 up, 8 in 2024-03-21T13:00:22.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:22 smithi043 ceph-mon[41699]: osdmap e396: 8 total, 8 up, 8 in 2024-03-21T13:00:23.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:23 smithi155 ceph-mon[33846]: pgmap v570: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:00:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:23 smithi043 ceph-mon[30062]: pgmap v570: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:00:23.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:23 smithi043 ceph-mon[41699]: pgmap v570: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:00:25.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:25 smithi155 ceph-mon[33846]: pgmap v571: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-21T13:00:25.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:25 smithi043 ceph-mon[30062]: pgmap v571: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-21T13:00:25.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:25 smithi043 ceph-mon[41699]: pgmap v571: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-21T13:00:27.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:27 smithi155 ceph-mon[33846]: pgmap v572: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-21T13:00:27.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:27 smithi155 ceph-mon[33846]: osdmap e397: 8 total, 8 up, 8 in 2024-03-21T13:00:27.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:27 smithi043 ceph-mon[30062]: pgmap v572: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-21T13:00:27.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:27 smithi043 ceph-mon[30062]: osdmap e397: 8 total, 8 up, 8 in 2024-03-21T13:00:27.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:27 smithi043 ceph-mon[41699]: pgmap v572: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-21T13:00:27.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:27 smithi043 ceph-mon[41699]: osdmap e397: 8 total, 8 up, 8 in 2024-03-21T13:00:28.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:00:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:00:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:00:28.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:00:28 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:00:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:29 smithi155 ceph-mon[33846]: pgmap v574: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:00:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:29 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:00:29.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:29 smithi043 ceph-mon[30062]: pgmap v574: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:00:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:29 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:00:29.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:29 smithi043 ceph-mon[41699]: pgmap v574: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:00:29.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:29 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:00:31.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:31 smithi155 ceph-mon[33846]: pgmap v575: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-21T13:00:31.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:31 smithi043 ceph-mon[30062]: pgmap v575: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-21T13:00:31.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:31 smithi043 ceph-mon[41699]: pgmap v575: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-21T13:00:32.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:32 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:00:32.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:32 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:00:32.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:32 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-63"}]: dispatch 2024-03-21T13:00:32.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:32 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-63"}]: dispatch 2024-03-21T13:00:32.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:32 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:00:32.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:32 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:00:32.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:32 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-63"}]: dispatch 2024-03-21T13:00:32.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:32 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-63"}]: dispatch 2024-03-21T13:00:32.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:32 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:00:32.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:32 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:00:32.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:32 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-63"}]: dispatch 2024-03-21T13:00:32.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:32 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-63"}]: dispatch 2024-03-21T13:00:33.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:33 smithi155 ceph-mon[33846]: pgmap v576: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-21T13:00:33.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:33 smithi155 ceph-mon[33846]: osdmap e398: 8 total, 8 up, 8 in 2024-03-21T13:00:33.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:33 smithi043 ceph-mon[30062]: pgmap v576: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-21T13:00:33.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:33 smithi043 ceph-mon[30062]: osdmap e398: 8 total, 8 up, 8 in 2024-03-21T13:00:33.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:33 smithi043 ceph-mon[41699]: pgmap v576: 356 pgs: 356 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-21T13:00:33.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:33 smithi043 ceph-mon[41699]: osdmap e398: 8 total, 8 up, 8 in 2024-03-21T13:00:34.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:34 smithi155 ceph-mon[33846]: osdmap e399: 8 total, 8 up, 8 in 2024-03-21T13:00:34.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:34 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:00:34.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:34 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:00:34.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:34 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:00:34.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:34 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:00:34.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:34 smithi155 ceph-mon[33846]: osdmap e400: 8 total, 8 up, 8 in 2024-03-21T13:00:34.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:34 smithi043 ceph-mon[30062]: osdmap e399: 8 total, 8 up, 8 in 2024-03-21T13:00:34.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:34 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:00:34.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:34 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:00:34.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:34 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:00:34.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:34 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:00:34.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:34 smithi043 ceph-mon[30062]: osdmap e400: 8 total, 8 up, 8 in 2024-03-21T13:00:34.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:34 smithi043 ceph-mon[41699]: osdmap e399: 8 total, 8 up, 8 in 2024-03-21T13:00:34.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:34 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:00:34.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:34 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:00:34.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:34 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:00:34.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:34 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:00:34.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:34 smithi043 ceph-mon[41699]: osdmap e400: 8 total, 8 up, 8 in 2024-03-21T13:00:35.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:35 smithi155 ceph-mon[33846]: pgmap v579: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:00:35.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:35 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:00:35.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:35 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:00:35.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:35 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:00:35.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:35 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:00:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:35 smithi043 ceph-mon[30062]: pgmap v579: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:00:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:35 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:00:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:35 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:00:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:35 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:00:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:35 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:00:35.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:35 smithi043 ceph-mon[41699]: pgmap v579: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:00:35.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:35 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:00:35.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:35 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:00:35.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:35 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:00:35.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:35 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:00:36.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:36 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:00:36.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:36 smithi155 ceph-mon[33846]: osdmap e401: 8 total, 8 up, 8 in 2024-03-21T13:00:36.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:36 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:00:36.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:36 smithi043 ceph-mon[30062]: osdmap e401: 8 total, 8 up, 8 in 2024-03-21T13:00:36.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:36 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:00:36.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:36 smithi043 ceph-mon[41699]: osdmap e401: 8 total, 8 up, 8 in 2024-03-21T13:00:37.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:37 smithi155 ceph-mon[33846]: pgmap v582: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:00:37.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:37 smithi155 ceph-mon[33846]: osdmap e402: 8 total, 8 up, 8 in 2024-03-21T13:00:37.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:37 smithi043 ceph-mon[30062]: pgmap v582: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:00:37.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:37 smithi043 ceph-mon[30062]: osdmap e402: 8 total, 8 up, 8 in 2024-03-21T13:00:37.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:37 smithi043 ceph-mon[41699]: pgmap v582: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:00:37.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:37 smithi043 ceph-mon[41699]: osdmap e402: 8 total, 8 up, 8 in 2024-03-21T13:00:38.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:00:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:00:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:00:38.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:38 smithi155 ceph-mon[33846]: osdmap e403: 8 total, 8 up, 8 in 2024-03-21T13:00:38.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:00:38 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:00:38.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:38 smithi043 ceph-mon[30062]: osdmap e403: 8 total, 8 up, 8 in 2024-03-21T13:00:38.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:38 smithi043 ceph-mon[41699]: osdmap e403: 8 total, 8 up, 8 in 2024-03-21T13:00:39.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:39 smithi155 ceph-mon[33846]: pgmap v585: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:00:39.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:39 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:00:39.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:39 smithi155 ceph-mon[33846]: osdmap e404: 8 total, 8 up, 8 in 2024-03-21T13:00:39.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:39 smithi043 ceph-mon[30062]: pgmap v585: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:00:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:39 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:00:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:39 smithi043 ceph-mon[30062]: osdmap e404: 8 total, 8 up, 8 in 2024-03-21T13:00:39.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:39 smithi043 ceph-mon[41699]: pgmap v585: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 666 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:00:39.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:39 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:00:39.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:39 smithi043 ceph-mon[41699]: osdmap e404: 8 total, 8 up, 8 in 2024-03-21T13:00:41.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:41 smithi155 ceph-mon[33846]: pgmap v587: 356 pgs: 356 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-03-21T13:00:41.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:41 smithi043 ceph-mon[30062]: pgmap v587: 356 pgs: 356 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-03-21T13:00:41.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:41 smithi043 ceph-mon[41699]: pgmap v587: 356 pgs: 356 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-03-21T13:00:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:42 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:00:42.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:42 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:00:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:42 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:00:43.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:43 smithi043 ceph-mon[30062]: pgmap v588: 356 pgs: 356 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-03-21T13:00:43.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:43 smithi043 ceph-mon[41699]: pgmap v588: 356 pgs: 356 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-03-21T13:00:44.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:43 smithi155 ceph-mon[33846]: pgmap v588: 356 pgs: 356 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-03-21T13:00:45.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:45 smithi043 ceph-mon[30062]: pgmap v589: 356 pgs: 356 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-03-21T13:00:45.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:45 smithi043 ceph-mon[41699]: pgmap v589: 356 pgs: 356 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-03-21T13:00:46.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:45 smithi155 ceph-mon[33846]: pgmap v589: 356 pgs: 356 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-03-21T13:00:47.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:47 smithi043 ceph-mon[30062]: pgmap v590: 356 pgs: 356 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-21T13:00:47.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:47 smithi043 ceph-mon[30062]: osdmap e405: 8 total, 8 up, 8 in 2024-03-21T13:00:47.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:47 smithi043 ceph-mon[41699]: pgmap v590: 356 pgs: 356 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-21T13:00:47.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:47 smithi043 ceph-mon[41699]: osdmap e405: 8 total, 8 up, 8 in 2024-03-21T13:00:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:47 smithi155 ceph-mon[33846]: pgmap v590: 356 pgs: 356 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.2 KiB/s wr, 5 op/s 2024-03-21T13:00:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:47 smithi155 ceph-mon[33846]: osdmap e405: 8 total, 8 up, 8 in 2024-03-21T13:00:48.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:00:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:00:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:00:48.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:00:48 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:00:49.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:49 smithi043 ceph-mon[30062]: pgmap v592: 356 pgs: 356 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 987 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:00:49.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:49 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:00:49.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:49 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:00:49.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:49 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:00:49.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:49 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-65"}]: dispatch 2024-03-21T13:00:49.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:49 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-65"}]: dispatch 2024-03-21T13:00:49.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:49 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:00:49.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:49 smithi043 ceph-mon[41699]: pgmap v592: 356 pgs: 356 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 987 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:00:49.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:49 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:00:49.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:49 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:00:49.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:49 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:00:49.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:49 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-65"}]: dispatch 2024-03-21T13:00:49.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:49 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-65"}]: dispatch 2024-03-21T13:00:49.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:49 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:00:50.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:49 smithi155 ceph-mon[33846]: pgmap v592: 356 pgs: 356 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail; 987 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:00:50.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:49 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:00:50.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:49 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:00:50.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:49 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:00:50.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:49 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-65"}]: dispatch 2024-03-21T13:00:50.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:49 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-65"}]: dispatch 2024-03-21T13:00:50.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:49 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:00:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:50 smithi155 ceph-mon[33846]: osdmap e406: 8 total, 8 up, 8 in 2024-03-21T13:00:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:50 smithi043 ceph-mon[30062]: osdmap e406: 8 total, 8 up, 8 in 2024-03-21T13:00:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:50 smithi043 ceph-mon[41699]: osdmap e406: 8 total, 8 up, 8 in 2024-03-21T13:00:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:51 smithi155 ceph-mon[33846]: pgmap v594: 324 pgs: 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:00:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:51 smithi155 ceph-mon[33846]: osdmap e407: 8 total, 8 up, 8 in 2024-03-21T13:00:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:51 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:00:52.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:51 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:00:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:51 smithi043 ceph-mon[30062]: pgmap v594: 324 pgs: 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:00:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:51 smithi043 ceph-mon[30062]: osdmap e407: 8 total, 8 up, 8 in 2024-03-21T13:00:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:51 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:00:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:51 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:00:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:51 smithi043 ceph-mon[41699]: pgmap v594: 324 pgs: 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:00:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:51 smithi043 ceph-mon[41699]: osdmap e407: 8 total, 8 up, 8 in 2024-03-21T13:00:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:51 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:00:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:51 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:00:53.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:52 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:00:53.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:52 smithi155 ceph-mon[33846]: osdmap e408: 8 total, 8 up, 8 in 2024-03-21T13:00:53.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:00:53.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:52 smithi155 ceph-mon[33846]: pgmap v597: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:00:53.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:52 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:00:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:52 smithi043 ceph-mon[30062]: osdmap e408: 8 total, 8 up, 8 in 2024-03-21T13:00:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:00:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:52 smithi043 ceph-mon[30062]: pgmap v597: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:00:53.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:52 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:00:53.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:52 smithi043 ceph-mon[41699]: osdmap e408: 8 total, 8 up, 8 in 2024-03-21T13:00:53.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:00:53.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:52 smithi043 ceph-mon[41699]: pgmap v597: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 659 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:00:54.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:54 smithi155 ceph-mon[33846]: osdmap e409: 8 total, 8 up, 8 in 2024-03-21T13:00:54.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:54 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:00:54.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:54 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:00:54.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:54 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-67"}]: dispatch 2024-03-21T13:00:54.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:54 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-67"}]: dispatch 2024-03-21T13:00:54.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:54 smithi043 ceph-mon[30062]: osdmap e409: 8 total, 8 up, 8 in 2024-03-21T13:00:54.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:54 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:00:54.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:54 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:00:54.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:54 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-67"}]: dispatch 2024-03-21T13:00:54.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:54 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-67"}]: dispatch 2024-03-21T13:00:54.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:54 smithi043 ceph-mon[41699]: osdmap e409: 8 total, 8 up, 8 in 2024-03-21T13:00:54.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:54 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:00:54.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:54 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:00:54.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:54 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-67"}]: dispatch 2024-03-21T13:00:54.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:54 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-67"}]: dispatch 2024-03-21T13:00:55.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:55 smithi155 ceph-mon[33846]: osdmap e410: 8 total, 8 up, 8 in 2024-03-21T13:00:55.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:55 smithi155 ceph-mon[33846]: pgmap v600: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:00:55.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:55 smithi155 ceph-mon[33846]: osdmap e411: 8 total, 8 up, 8 in 2024-03-21T13:00:55.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:55 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:00:55.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:55 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:00:55.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:55 smithi043 ceph-mon[30062]: osdmap e410: 8 total, 8 up, 8 in 2024-03-21T13:00:55.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:55 smithi043 ceph-mon[30062]: pgmap v600: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:00:55.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:55 smithi043 ceph-mon[30062]: osdmap e411: 8 total, 8 up, 8 in 2024-03-21T13:00:55.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:55 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:00:55.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:55 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:00:55.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:55 smithi043 ceph-mon[41699]: osdmap e410: 8 total, 8 up, 8 in 2024-03-21T13:00:55.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:55 smithi043 ceph-mon[41699]: pgmap v600: 324 pgs: 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:00:55.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:55 smithi043 ceph-mon[41699]: osdmap e411: 8 total, 8 up, 8 in 2024-03-21T13:00:55.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:55 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:00:55.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:55 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:00:56.598 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552533b80 2024-03-21T13:00:56.598 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525330d0 2024-03-21T13:00:56.598 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552533200 2024-03-21T13:00:56.599 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552533330 2024-03-21T13:00:56.599 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552533cb0 2024-03-21T13:00:56.599 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525336d0 2024-03-21T13:00:56.599 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552533800 2024-03-21T13:00:56.599 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552533950 2024-03-21T13:00:56.599 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552533de0 2024-03-21T13:00:56.599 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525342b0 2024-03-21T13:00:56.599 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525343e0 2024-03-21T13:00:56.599 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552531c50 2024-03-21T13:00:56.599 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552531d80 2024-03-21T13:00:56.599 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552531eb0 2024-03-21T13:00:56.599 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552534510 2024-03-21T13:00:56.599 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552534a00 2024-03-21T13:00:56.599 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552534b30 2024-03-21T13:00:56.599 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552533ef0 2024-03-21T13:00:56.600 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552534040 2024-03-21T13:00:56.600 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552534170 2024-03-21T13:00:56.600 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552534c60 2024-03-21T13:00:56.600 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552535150 2024-03-21T13:00:56.600 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552535280 2024-03-21T13:00:56.600 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552535860 2024-03-21T13:00:56.600 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552534db0 2024-03-21T13:00:56.600 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552534f00 2024-03-21T13:00:56.600 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552535030 2024-03-21T13:00:56.600 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552535970 2024-03-21T13:00:56.600 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525353b0 2024-03-21T13:00:56.600 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525354e0 2024-03-21T13:00:56.600 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552535650 2024-03-21T13:00:56.600 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552535aa0 2024-03-21T13:00:56.601 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552535fc0 2024-03-21T13:00:56.601 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552536140 2024-03-21T13:00:56.601 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552536720 2024-03-21T13:00:56.601 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552535bd0 2024-03-21T13:00:56.601 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552535ce0 2024-03-21T13:00:56.601 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552535df0 2024-03-21T13:00:56.601 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552536850 2024-03-21T13:00:56.601 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552536270 2024-03-21T13:00:56.601 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525363a0 2024-03-21T13:00:56.601 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525364d0 2024-03-21T13:00:56.601 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552536600 2024-03-21T13:00:56.601 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552536e30 2024-03-21T13:00:56.601 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552536980 2024-03-21T13:00:56.601 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552536ad0 2024-03-21T13:00:56.601 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552536c00 2024-03-21T13:00:56.601 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552536f60 2024-03-21T13:00:56.602 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552537430 2024-03-21T13:00:56.602 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552537560 2024-03-21T13:00:56.602 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552537b40 2024-03-21T13:00:56.602 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552537090 2024-03-21T13:00:56.602 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525371c0 2024-03-21T13:00:56.602 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525372f0 2024-03-21T13:00:56.602 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552537c70 2024-03-21T13:00:56.602 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552537690 2024-03-21T13:00:56.602 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525377c0 2024-03-21T13:00:56.602 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525378f0 2024-03-21T13:00:56.602 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552537a20 2024-03-21T13:00:56.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552538250 2024-03-21T13:00:56.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552537da0 2024-03-21T13:00:56.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552537ef0 2024-03-21T13:00:56.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552538020 2024-03-21T13:00:56.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552538380 2024-03-21T13:00:56.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552538850 2024-03-21T13:00:56.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552538980 2024-03-21T13:00:56.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552534640 2024-03-21T13:00:56.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552534770 2024-03-21T13:00:56.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525348c0 2024-03-21T13:00:56.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552538ab0 2024-03-21T13:00:56.603 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552538fa0 2024-03-21T13:00:56.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525390f0 2024-03-21T13:00:56.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552538130 2024-03-21T13:00:56.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552538be0 2024-03-21T13:00:56.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552538d10 2024-03-21T13:00:56.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552538e80 2024-03-21T13:00:56.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525396d0 2024-03-21T13:00:56.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552539220 2024-03-21T13:00:56.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552539370 2024-03-21T13:00:56.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525394a0 2024-03-21T13:00:56.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552539800 2024-03-21T13:00:56.604 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552539cd0 2024-03-21T13:00:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:56 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:00:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:56 smithi155 ceph-mon[33846]: osdmap e412: 8 total, 8 up, 8 in 2024-03-21T13:00:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:56 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:00:57.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:56 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:00:57.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:56 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:00:57.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:56 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-69"}]: dispatch 2024-03-21T13:00:57.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:56 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-69"}]: dispatch 2024-03-21T13:00:57.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:56 smithi155 ceph-mon[33846]: pgmap v603: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:00:57.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:56 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:00:57.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:56 smithi043 ceph-mon[30062]: osdmap e412: 8 total, 8 up, 8 in 2024-03-21T13:00:57.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:56 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:00:57.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:56 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:00:57.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:56 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:00:57.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:56 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-69"}]: dispatch 2024-03-21T13:00:57.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:56 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-69"}]: dispatch 2024-03-21T13:00:57.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:56 smithi043 ceph-mon[30062]: pgmap v603: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:00:57.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:56 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:00:57.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:56 smithi043 ceph-mon[41699]: osdmap e412: 8 total, 8 up, 8 in 2024-03-21T13:00:57.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:56 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:00:57.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:56 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:00:57.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:56 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:00:57.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:56 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-69"}]: dispatch 2024-03-21T13:00:57.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:56 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-69"}]: dispatch 2024-03-21T13:00:57.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:56 smithi043 ceph-mon[41699]: pgmap v603: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:00:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:57 smithi155 ceph-mon[33846]: osdmap e413: 8 total, 8 up, 8 in 2024-03-21T13:00:58.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:57 smithi043 ceph-mon[30062]: osdmap e413: 8 total, 8 up, 8 in 2024-03-21T13:00:58.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:00:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:00:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:00:58.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:57 smithi043 ceph-mon[41699]: osdmap e413: 8 total, 8 up, 8 in 2024-03-21T13:00:58.792 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:00:58 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:00:59.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:58 smithi155 ceph-mon[33846]: osdmap e414: 8 total, 8 up, 8 in 2024-03-21T13:00:59.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:58 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:00:59.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:58 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:00:59.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:58 smithi155 ceph-mon[33846]: pgmap v606: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:00:59.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:58 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:00:59.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:58 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:00:59.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:58 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:00:59.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:58 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:00:59.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:58 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:00:59.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:58 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:00:59.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:58 smithi155 ceph-mon[33846]: osdmap e415: 8 total, 8 up, 8 in 2024-03-21T13:00:59.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:58 smithi043 ceph-mon[30062]: osdmap e414: 8 total, 8 up, 8 in 2024-03-21T13:00:59.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:58 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:00:59.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:58 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:00:59.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:58 smithi043 ceph-mon[30062]: pgmap v606: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:00:59.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:58 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:00:59.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:58 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:00:59.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:58 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:00:59.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:58 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:00:59.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:58 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:00:59.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:58 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:00:59.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:58 smithi043 ceph-mon[30062]: osdmap e415: 8 total, 8 up, 8 in 2024-03-21T13:00:59.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:58 smithi043 ceph-mon[41699]: osdmap e414: 8 total, 8 up, 8 in 2024-03-21T13:00:59.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:58 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:00:59.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:58 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:00:59.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:58 smithi043 ceph-mon[41699]: pgmap v606: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 664 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:00:59.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:58 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:00:59.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:58 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:00:59.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:58 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:00:59.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:58 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:00:59.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:58 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:00:59.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:58 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:00:59.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:58 smithi043 ceph-mon[41699]: osdmap e415: 8 total, 8 up, 8 in 2024-03-21T13:01:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:59 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:01:00.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:59 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:01:00.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:59 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:01:00.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:59 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:01:00.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:59 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:01:00.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:00:59 smithi155 ceph-mon[33846]: osdmap e416: 8 total, 8 up, 8 in 2024-03-21T13:01:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:59 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:01:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:59 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:01:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:59 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:01:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:59 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:01:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:59 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:01:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:00:59 smithi043 ceph-mon[30062]: osdmap e416: 8 total, 8 up, 8 in 2024-03-21T13:01:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:59 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:01:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:59 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:01:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:59 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:01:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:59 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:01:00.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:59 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:01:00.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:00:59 smithi043 ceph-mon[41699]: osdmap e416: 8 total, 8 up, 8 in 2024-03-21T13:01:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:00 smithi155 ceph-mon[33846]: pgmap v609: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:01:01.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:00 smithi043 ceph-mon[30062]: pgmap v609: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:01:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:00 smithi043 ceph-mon[41699]: pgmap v609: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:01:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:01 smithi155 ceph-mon[33846]: osdmap e417: 8 total, 8 up, 8 in 2024-03-21T13:01:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:01 smithi043 ceph-mon[30062]: osdmap e417: 8 total, 8 up, 8 in 2024-03-21T13:01:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:01 smithi043 ceph-mon[41699]: osdmap e417: 8 total, 8 up, 8 in 2024-03-21T13:01:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:02 smithi155 ceph-mon[33846]: pgmap v611: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 719 B/s wr, 2 op/s 2024-03-21T13:01:03.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:02 smithi043 ceph-mon[30062]: pgmap v611: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 719 B/s wr, 2 op/s 2024-03-21T13:01:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:02 smithi043 ceph-mon[41699]: pgmap v611: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 652 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 719 B/s wr, 2 op/s 2024-03-21T13:01:05.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:05 smithi155 ceph-mon[33846]: pgmap v612: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-03-21T13:01:05.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:05 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:01:05.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:05 smithi043 ceph-mon[30062]: pgmap v612: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-03-21T13:01:05.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:05 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:01:05.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:05 smithi043 ceph-mon[41699]: pgmap v612: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.7 KiB/s wr, 6 op/s 2024-03-21T13:01:05.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:05 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:01:07.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:07 smithi155 ceph-mon[33846]: pgmap v613: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 844 B/s rd, 1.8 KiB/s wr, 4 op/s 2024-03-21T13:01:07.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:07 smithi043 ceph-mon[30062]: pgmap v613: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 844 B/s rd, 1.8 KiB/s wr, 4 op/s 2024-03-21T13:01:07.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:07 smithi043 ceph-mon[41699]: pgmap v613: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 844 B/s rd, 1.8 KiB/s wr, 4 op/s 2024-03-21T13:01:08.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:08 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:01:08.386 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:01:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:01:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:01:08.386 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:08 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:01:08.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:08 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:01:08.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:01:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:01:09.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:09 smithi155 ceph-mon[33846]: pgmap v614: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 742 B/s rd, 1.6 KiB/s wr, 3 op/s 2024-03-21T13:01:09.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:09 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:01:09.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:09 smithi043 ceph-mon[30062]: pgmap v614: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 742 B/s rd, 1.6 KiB/s wr, 3 op/s 2024-03-21T13:01:09.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:09 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:01:09.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:09 smithi043 ceph-mon[41699]: pgmap v614: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 742 B/s rd, 1.6 KiB/s wr, 3 op/s 2024-03-21T13:01:09.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:09 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:01:11.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:11 smithi155 ceph-mon[33846]: pgmap v615: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-03-21T13:01:11.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:11 smithi043 ceph-mon[30062]: pgmap v615: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-03-21T13:01:11.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:11 smithi043 ceph-mon[41699]: pgmap v615: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-03-21T13:01:13.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:13 smithi155 ceph-mon[33846]: pgmap v616: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1002 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-21T13:01:13.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:13 smithi043 ceph-mon[30062]: pgmap v616: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1002 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-21T13:01:13.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:13 smithi043 ceph-mon[41699]: pgmap v616: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1002 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-21T13:01:15.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:15 smithi155 ceph-mon[33846]: pgmap v617: 356 pgs: 356 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-03-21T13:01:15.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:15 smithi043 ceph-mon[30062]: pgmap v617: 356 pgs: 356 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-03-21T13:01:15.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:15 smithi043 ceph-mon[41699]: pgmap v617: 356 pgs: 356 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-03-21T13:01:17.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:17 smithi155 ceph-mon[33846]: pgmap v618: 356 pgs: 356 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:01:17.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:17 smithi043 ceph-mon[30062]: pgmap v618: 356 pgs: 356 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:01:17.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:17 smithi043 ceph-mon[41699]: pgmap v618: 356 pgs: 356 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:01:18.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:01:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:01:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:01:18.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:01:18 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:01:19.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:19 smithi155 ceph-mon[33846]: pgmap v619: 356 pgs: 356 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:01:19.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:19 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:01:19.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:19 smithi043 ceph-mon[30062]: pgmap v619: 356 pgs: 356 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:01:19.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:19 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:01:19.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:19 smithi043 ceph-mon[41699]: pgmap v619: 356 pgs: 356 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:01:19.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:19 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:01:20.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:20 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:01:20.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:20 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:01:20.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:20 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:01:21.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:21 smithi155 ceph-mon[33846]: pgmap v620: 356 pgs: 356 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:01:21.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:21 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:01:21.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:21 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:01:21.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:21 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-71"}]: dispatch 2024-03-21T13:01:21.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:21 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-71"}]: dispatch 2024-03-21T13:01:21.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:21 smithi043 ceph-mon[30062]: pgmap v620: 356 pgs: 356 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:01:21.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:21 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:01:21.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:21 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:01:21.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:21 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-71"}]: dispatch 2024-03-21T13:01:21.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:21 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-71"}]: dispatch 2024-03-21T13:01:21.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:21 smithi043 ceph-mon[41699]: pgmap v620: 356 pgs: 356 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:01:21.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:21 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:01:21.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:21 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:01:21.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:21 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-71"}]: dispatch 2024-03-21T13:01:21.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:21 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-71"}]: dispatch 2024-03-21T13:01:22.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:22 smithi155 ceph-mon[33846]: osdmap e418: 8 total, 8 up, 8 in 2024-03-21T13:01:22.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:22 smithi043 ceph-mon[30062]: osdmap e418: 8 total, 8 up, 8 in 2024-03-21T13:01:22.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:22 smithi043 ceph-mon[41699]: osdmap e418: 8 total, 8 up, 8 in 2024-03-21T13:01:23.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:23 smithi155 ceph-mon[33846]: pgmap v622: 324 pgs: 324 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-03-21T13:01:23.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:23 smithi155 ceph-mon[33846]: osdmap e419: 8 total, 8 up, 8 in 2024-03-21T13:01:23.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:23 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:01:23.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:23 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:01:23.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:23 smithi043 ceph-mon[30062]: pgmap v622: 324 pgs: 324 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-03-21T13:01:23.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:23 smithi043 ceph-mon[30062]: osdmap e419: 8 total, 8 up, 8 in 2024-03-21T13:01:23.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:23 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:01:23.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:23 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:01:23.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:23 smithi043 ceph-mon[41699]: pgmap v622: 324 pgs: 324 active+clean; 8.4 MiB data, 632 MiB used, 715 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-03-21T13:01:23.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:23 smithi043 ceph-mon[41699]: osdmap e419: 8 total, 8 up, 8 in 2024-03-21T13:01:23.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:23 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:01:23.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:23 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:01:24.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:24 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:01:24.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:24 smithi155 ceph-mon[33846]: osdmap e420: 8 total, 8 up, 8 in 2024-03-21T13:01:24.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:24 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:01:24.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:24 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:01:24.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:24 smithi043 ceph-mon[30062]: osdmap e420: 8 total, 8 up, 8 in 2024-03-21T13:01:24.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:24 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:01:24.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:24 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:01:24.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:24 smithi043 ceph-mon[41699]: osdmap e420: 8 total, 8 up, 8 in 2024-03-21T13:01:24.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:24 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:01:25.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:25 smithi155 ceph-mon[33846]: pgmap v625: 356 pgs: 6 creating+activating, 25 creating+peering, 325 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:01:25.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:25 smithi155 ceph-mon[33846]: osdmap e421: 8 total, 8 up, 8 in 2024-03-21T13:01:25.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:25 smithi043 ceph-mon[30062]: pgmap v625: 356 pgs: 6 creating+activating, 25 creating+peering, 325 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:01:25.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:25 smithi043 ceph-mon[30062]: osdmap e421: 8 total, 8 up, 8 in 2024-03-21T13:01:25.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:25 smithi043 ceph-mon[41699]: pgmap v625: 356 pgs: 6 creating+activating, 25 creating+peering, 325 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:01:25.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:25 smithi043 ceph-mon[41699]: osdmap e421: 8 total, 8 up, 8 in 2024-03-21T13:01:26.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:26 smithi155 ceph-mon[33846]: osdmap e422: 8 total, 8 up, 8 in 2024-03-21T13:01:26.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:26 smithi043 ceph-mon[30062]: osdmap e422: 8 total, 8 up, 8 in 2024-03-21T13:01:26.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:26 smithi043 ceph-mon[41699]: osdmap e422: 8 total, 8 up, 8 in 2024-03-21T13:01:27.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:27 smithi155 ceph-mon[33846]: pgmap v628: 356 pgs: 6 creating+activating, 25 creating+peering, 325 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:01:27.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:27 smithi043 ceph-mon[30062]: pgmap v628: 356 pgs: 6 creating+activating, 25 creating+peering, 325 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:01:27.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:27 smithi043 ceph-mon[41699]: pgmap v628: 356 pgs: 6 creating+activating, 25 creating+peering, 325 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:01:28.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:01:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:01:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:01:28.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:01:28 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:01:29.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:29 smithi155 ceph-mon[33846]: pgmap v629: 356 pgs: 6 creating+activating, 25 creating+peering, 325 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 865 B/s rd, 0 op/s 2024-03-21T13:01:29.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:29 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:01:29.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:29 smithi043 ceph-mon[30062]: pgmap v629: 356 pgs: 6 creating+activating, 25 creating+peering, 325 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 865 B/s rd, 0 op/s 2024-03-21T13:01:29.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:29 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:01:29.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:29 smithi043 ceph-mon[41699]: pgmap v629: 356 pgs: 6 creating+activating, 25 creating+peering, 325 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 865 B/s rd, 0 op/s 2024-03-21T13:01:29.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:29 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:01:31.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:31 smithi155 ceph-mon[33846]: pgmap v630: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-03-21T13:01:31.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:31 smithi043 ceph-mon[30062]: pgmap v630: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-03-21T13:01:31.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:31 smithi043 ceph-mon[41699]: pgmap v630: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.0 KiB/s wr, 4 op/s 2024-03-21T13:01:33.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:33 smithi155 ceph-mon[33846]: pgmap v631: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 895 B/s wr, 3 op/s 2024-03-21T13:01:33.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:33 smithi043 ceph-mon[30062]: pgmap v631: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 895 B/s wr, 3 op/s 2024-03-21T13:01:33.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:33 smithi043 ceph-mon[41699]: pgmap v631: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 895 B/s wr, 3 op/s 2024-03-21T13:01:35.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:35 smithi155 ceph-mon[33846]: pgmap v632: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 723 B/s wr, 3 op/s 2024-03-21T13:01:35.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:01:35.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:35 smithi043 ceph-mon[30062]: pgmap v632: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 723 B/s wr, 3 op/s 2024-03-21T13:01:35.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:01:35.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:35 smithi043 ceph-mon[41699]: pgmap v632: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 723 B/s wr, 3 op/s 2024-03-21T13:01:35.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:01:36.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:01:36.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:36 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:01:36.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-73"}]: dispatch 2024-03-21T13:01:36.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:36 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-73"}]: dispatch 2024-03-21T13:01:36.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:01:36.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:36 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:01:36.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-73"}]: dispatch 2024-03-21T13:01:36.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:36 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-73"}]: dispatch 2024-03-21T13:01:36.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:01:36.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:36 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:01:36.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-73"}]: dispatch 2024-03-21T13:01:36.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:36 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-73"}]: dispatch 2024-03-21T13:01:37.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:37 smithi155 ceph-mon[33846]: pgmap v633: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 657 B/s wr, 2 op/s 2024-03-21T13:01:37.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:37 smithi155 ceph-mon[33846]: osdmap e423: 8 total, 8 up, 8 in 2024-03-21T13:01:37.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:37 smithi043 ceph-mon[30062]: pgmap v633: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 657 B/s wr, 2 op/s 2024-03-21T13:01:37.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:37 smithi043 ceph-mon[30062]: osdmap e423: 8 total, 8 up, 8 in 2024-03-21T13:01:37.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:37 smithi043 ceph-mon[41699]: pgmap v633: 356 pgs: 356 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 657 B/s wr, 2 op/s 2024-03-21T13:01:37.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:37 smithi043 ceph-mon[41699]: osdmap e423: 8 total, 8 up, 8 in 2024-03-21T13:01:38.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:38 smithi043 ceph-mon[30062]: osdmap e424: 8 total, 8 up, 8 in 2024-03-21T13:01:38.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:38 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:01:38.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:38 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:01:38.386 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:01:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:01:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:01:38.386 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:38 smithi043 ceph-mon[41699]: osdmap e424: 8 total, 8 up, 8 in 2024-03-21T13:01:38.386 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:38 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:01:38.386 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:38 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:01:38.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:38 smithi155 ceph-mon[33846]: osdmap e424: 8 total, 8 up, 8 in 2024-03-21T13:01:38.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:38 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:01:38.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:38 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:01:38.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:01:38 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:01:39.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:39 smithi155 ceph-mon[33846]: pgmap v636: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:01:39.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:39 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:01:39.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:39 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:01:39.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:39 smithi155 ceph-mon[33846]: osdmap e425: 8 total, 8 up, 8 in 2024-03-21T13:01:39.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:39 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:01:39.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:39 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:01:39.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:39 smithi043 ceph-mon[41699]: pgmap v636: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:01:39.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:39 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:01:39.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:39 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:01:39.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:39 smithi043 ceph-mon[41699]: osdmap e425: 8 total, 8 up, 8 in 2024-03-21T13:01:39.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:39 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:01:39.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:39 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:01:39.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:39 smithi043 ceph-mon[30062]: pgmap v636: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:01:39.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:39 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:01:39.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:39 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:01:39.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:39 smithi043 ceph-mon[30062]: osdmap e425: 8 total, 8 up, 8 in 2024-03-21T13:01:39.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:39 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:01:39.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:39 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:01:40.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:40 smithi155 ceph-mon[33846]: osdmap e426: 8 total, 8 up, 8 in 2024-03-21T13:01:40.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:40 smithi043 ceph-mon[30062]: osdmap e426: 8 total, 8 up, 8 in 2024-03-21T13:01:40.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:40 smithi043 ceph-mon[41699]: osdmap e426: 8 total, 8 up, 8 in 2024-03-21T13:01:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:41 smithi043 ceph-mon[41699]: pgmap v639: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:01:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:41 smithi043 ceph-mon[41699]: osdmap e427: 8 total, 8 up, 8 in 2024-03-21T13:01:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:01:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:41 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:01:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-75"}]: dispatch 2024-03-21T13:01:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:41 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-75"}]: dispatch 2024-03-21T13:01:41.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:41 smithi043 ceph-mon[30062]: pgmap v639: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:01:41.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:41 smithi043 ceph-mon[30062]: osdmap e427: 8 total, 8 up, 8 in 2024-03-21T13:01:41.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:01:41.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:41 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:01:41.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-75"}]: dispatch 2024-03-21T13:01:41.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:41 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-75"}]: dispatch 2024-03-21T13:01:41.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:41 smithi155 ceph-mon[33846]: pgmap v639: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:01:41.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:41 smithi155 ceph-mon[33846]: osdmap e427: 8 total, 8 up, 8 in 2024-03-21T13:01:41.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:01:41.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:41 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:01:41.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-75"}]: dispatch 2024-03-21T13:01:41.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:41 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-75"}]: dispatch 2024-03-21T13:01:42.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:42 smithi043 ceph-mon[41699]: osdmap e428: 8 total, 8 up, 8 in 2024-03-21T13:01:42.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:42 smithi043 ceph-mon[41699]: osdmap e429: 8 total, 8 up, 8 in 2024-03-21T13:01:42.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:01:42.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:42 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:01:42.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:42 smithi043 ceph-mon[30062]: osdmap e428: 8 total, 8 up, 8 in 2024-03-21T13:01:42.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:42 smithi043 ceph-mon[30062]: osdmap e429: 8 total, 8 up, 8 in 2024-03-21T13:01:42.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:01:42.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:42 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:01:42.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:42 smithi155 ceph-mon[33846]: osdmap e428: 8 total, 8 up, 8 in 2024-03-21T13:01:42.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:42 smithi155 ceph-mon[33846]: osdmap e429: 8 total, 8 up, 8 in 2024-03-21T13:01:42.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:01:42.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:42 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:01:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:43 smithi043 ceph-mon[30062]: pgmap v642: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:01:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:43 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:01:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:43 smithi043 ceph-mon[30062]: osdmap e430: 8 total, 8 up, 8 in 2024-03-21T13:01:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:01:43.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:43 smithi043 ceph-mon[41699]: pgmap v642: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:01:43.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:43 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:01:43.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:43 smithi043 ceph-mon[41699]: osdmap e430: 8 total, 8 up, 8 in 2024-03-21T13:01:43.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:01:43.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:43 smithi155 ceph-mon[33846]: pgmap v642: 324 pgs: 324 active+clean; 8.4 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:01:43.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:43 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:01:43.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:43 smithi155 ceph-mon[33846]: osdmap e430: 8 total, 8 up, 8 in 2024-03-21T13:01:43.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:01:45.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:45 smithi043 ceph-mon[30062]: pgmap v645: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:01:45.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:45 smithi043 ceph-mon[30062]: osdmap e431: 8 total, 8 up, 8 in 2024-03-21T13:01:45.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:01:45.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:45 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:01:45.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-77"}]: dispatch 2024-03-21T13:01:45.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:45 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-77"}]: dispatch 2024-03-21T13:01:45.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:45 smithi043 ceph-mon[41699]: pgmap v645: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:01:45.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:45 smithi043 ceph-mon[41699]: osdmap e431: 8 total, 8 up, 8 in 2024-03-21T13:01:45.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:45 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:01:45.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:45 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:01:45.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:45 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-77"}]: dispatch 2024-03-21T13:01:45.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:45 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-77"}]: dispatch 2024-03-21T13:01:45.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:45 smithi155 ceph-mon[33846]: pgmap v645: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:01:45.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:45 smithi155 ceph-mon[33846]: osdmap e431: 8 total, 8 up, 8 in 2024-03-21T13:01:45.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:45 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:01:45.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:45 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:01:45.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:45 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-77"}]: dispatch 2024-03-21T13:01:45.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:45 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-77"}]: dispatch 2024-03-21T13:01:46.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:46 smithi043 ceph-mon[30062]: osdmap e432: 8 total, 8 up, 8 in 2024-03-21T13:01:46.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:46 smithi043 ceph-mon[41699]: osdmap e432: 8 total, 8 up, 8 in 2024-03-21T13:01:46.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:46 smithi155 ceph-mon[33846]: osdmap e432: 8 total, 8 up, 8 in 2024-03-21T13:01:47.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:47 smithi155 ceph-mon[33846]: pgmap v648: 324 pgs: 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:01:47.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:47 smithi155 ceph-mon[33846]: osdmap e433: 8 total, 8 up, 8 in 2024-03-21T13:01:47.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:01:47.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:47 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:01:47.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:47 smithi043 ceph-mon[41699]: pgmap v648: 324 pgs: 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:01:47.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:47 smithi043 ceph-mon[41699]: osdmap e433: 8 total, 8 up, 8 in 2024-03-21T13:01:47.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:01:47.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:47 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:01:47.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:47 smithi043 ceph-mon[30062]: pgmap v648: 324 pgs: 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:01:47.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:47 smithi043 ceph-mon[30062]: osdmap e433: 8 total, 8 up, 8 in 2024-03-21T13:01:47.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:01:47.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:47 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:01:48.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:48 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:01:48.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:48 smithi043 ceph-mon[30062]: osdmap e434: 8 total, 8 up, 8 in 2024-03-21T13:01:48.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:48 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:01:48.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:48 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:01:48.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:01:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:01:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:01:48.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:48 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:01:48.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:48 smithi043 ceph-mon[41699]: osdmap e434: 8 total, 8 up, 8 in 2024-03-21T13:01:48.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:48 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:01:48.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:48 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:01:48.559 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:48 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:01:48.559 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:48 smithi155 ceph-mon[33846]: osdmap e434: 8 total, 8 up, 8 in 2024-03-21T13:01:48.559 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:48 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:01:48.559 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:48 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:01:48.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:01:48 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:01:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:49 smithi155 ceph-mon[33846]: pgmap v651: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:01:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:49 smithi155 ceph-mon[33846]: osdmap e435: 8 total, 8 up, 8 in 2024-03-21T13:01:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:49 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:01:49.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:49 smithi043 ceph-mon[41699]: pgmap v651: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:01:49.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:49 smithi043 ceph-mon[41699]: osdmap e435: 8 total, 8 up, 8 in 2024-03-21T13:01:49.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:49 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:01:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:49 smithi043 ceph-mon[30062]: pgmap v651: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:01:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:49 smithi043 ceph-mon[30062]: osdmap e435: 8 total, 8 up, 8 in 2024-03-21T13:01:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:49 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:01:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:50 smithi155 ceph-mon[33846]: osdmap e436: 8 total, 8 up, 8 in 2024-03-21T13:01:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:50 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "293.1f"}]: dispatch 2024-03-21T13:01:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:50 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "293.1f"}]: dispatch 2024-03-21T13:01:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:50 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:01:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:50 smithi043 ceph-mon[30062]: osdmap e436: 8 total, 8 up, 8 in 2024-03-21T13:01:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:50 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "293.1f"}]: dispatch 2024-03-21T13:01:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:50 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "293.1f"}]: dispatch 2024-03-21T13:01:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:50 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:01:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:50 smithi043 ceph-mon[41699]: osdmap e436: 8 total, 8 up, 8 in 2024-03-21T13:01:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:50 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "293.1f"}]: dispatch 2024-03-21T13:01:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:50 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "293.1f"}]: dispatch 2024-03-21T13:01:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:50 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:01:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:51 smithi155 ceph-mon[33846]: 293.1f deep-scrub starts 2024-03-21T13:01:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:51 smithi155 ceph-mon[33846]: 293.1f deep-scrub ok 2024-03-21T13:01:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:51 smithi155 ceph-mon[33846]: pgmap v654: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s rd, 1.2 KiB/s wr, 181 op/s 2024-03-21T13:01:51.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:51 smithi043 ceph-mon[30062]: 293.1f deep-scrub starts 2024-03-21T13:01:51.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:51 smithi043 ceph-mon[30062]: 293.1f deep-scrub ok 2024-03-21T13:01:51.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:51 smithi043 ceph-mon[30062]: pgmap v654: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s rd, 1.2 KiB/s wr, 181 op/s 2024-03-21T13:01:51.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:51 smithi043 ceph-mon[41699]: 293.1f deep-scrub starts 2024-03-21T13:01:51.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:51 smithi043 ceph-mon[41699]: 293.1f deep-scrub ok 2024-03-21T13:01:51.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:51 smithi043 ceph-mon[41699]: pgmap v654: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 109 KiB/s rd, 1.2 KiB/s wr, 181 op/s 2024-03-21T13:01:53.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:53 smithi155 ceph-mon[33846]: pgmap v655: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s rd, 878 B/s wr, 124 op/s 2024-03-21T13:01:53.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:53 smithi043 ceph-mon[30062]: pgmap v655: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s rd, 878 B/s wr, 124 op/s 2024-03-21T13:01:53.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:53 smithi043 ceph-mon[41699]: pgmap v655: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s rd, 878 B/s wr, 124 op/s 2024-03-21T13:01:55.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:55 smithi155 ceph-mon[33846]: pgmap v656: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 1.0 KiB/s wr, 108 op/s 2024-03-21T13:01:55.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:55 smithi043 ceph-mon[30062]: pgmap v656: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 1.0 KiB/s wr, 108 op/s 2024-03-21T13:01:55.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:55 smithi043 ceph-mon[41699]: pgmap v656: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 1.0 KiB/s wr, 108 op/s 2024-03-21T13:01:57.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:57 smithi155 ceph-mon[33846]: pgmap v657: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 895 B/s wr, 92 op/s 2024-03-21T13:01:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:57 smithi043 ceph-mon[30062]: pgmap v657: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 895 B/s wr, 92 op/s 2024-03-21T13:01:57.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:57 smithi043 ceph-mon[41699]: pgmap v657: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 895 B/s wr, 92 op/s 2024-03-21T13:01:58.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:01:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:01:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:01:58.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:01:58 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:01:59.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:59 smithi155 ceph-mon[33846]: pgmap v658: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 521 B/s rd, 208 B/s wr, 1 op/s 2024-03-21T13:01:59.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:59 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:01:59.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:59 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:01:59.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:59 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:01:59.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:59 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:01:59.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:01:59 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:01:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:59 smithi043 ceph-mon[30062]: pgmap v658: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 521 B/s rd, 208 B/s wr, 1 op/s 2024-03-21T13:01:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:59 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:01:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:59 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:01:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:59 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:01:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:59 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:01:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:01:59 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:01:59.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:59 smithi043 ceph-mon[41699]: pgmap v658: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 521 B/s rd, 208 B/s wr, 1 op/s 2024-03-21T13:01:59.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:59 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:01:59.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:59 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:01:59.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:59 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:01:59.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:59 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:01:59.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:01:59 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:02:01.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:01 smithi155 ceph-mon[33846]: pgmap v659: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 947 B/s rd, 189 B/s wr, 1 op/s 2024-03-21T13:02:01.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:01 smithi043 ceph-mon[30062]: pgmap v659: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 947 B/s rd, 189 B/s wr, 1 op/s 2024-03-21T13:02:01.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:01 smithi043 ceph-mon[41699]: pgmap v659: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 947 B/s rd, 189 B/s wr, 1 op/s 2024-03-21T13:02:03.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:03 smithi155 ceph-mon[33846]: pgmap v660: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 170 B/s wr, 1 op/s 2024-03-21T13:02:03.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:03 smithi043 ceph-mon[30062]: pgmap v660: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 170 B/s wr, 1 op/s 2024-03-21T13:02:03.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:03 smithi043 ceph-mon[41699]: pgmap v660: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 170 B/s wr, 1 op/s 2024-03-21T13:02:04.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:04 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:02:04.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:04 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:02:04.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:04 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:02:05.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:05 smithi155 ceph-mon[33846]: pgmap v661: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-03-21T13:02:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:05 smithi043 ceph-mon[30062]: pgmap v661: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-03-21T13:02:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:05 smithi043 ceph-mon[41699]: pgmap v661: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 170 B/s wr, 1 op/s 2024-03-21T13:02:07.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:07 smithi155 ceph-mon[33846]: pgmap v662: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:02:07.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:07 smithi043 ceph-mon[30062]: pgmap v662: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:02:07.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:07 smithi043 ceph-mon[41699]: pgmap v662: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:02:08.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:02:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:02:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:02:08.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:02:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:02:09.638 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 13:02:09 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-2[58226]: 2024-03-21T13:02:09.287+0000 7f6a2179f700 -1 snap_mapper.add_oid found existing snaps mapped on 293:fbb8e8a0:test-rados-api-smithi043-81880-80::foo:2, removing 2024-03-21T13:02:09.638 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 13:02:09 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-4[39286]: 2024-03-21T13:02:09.288+0000 7f365a922700 -1 snap_mapper.add_oid found existing snaps mapped on 293:fbb8e8a0:test-rados-api-smithi043-81880-80::foo:2, removing 2024-03-21T13:02:09.639 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:02:09 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:02:09.288+0000 7f438e903700 -1 snap_mapper.add_oid found existing snaps mapped on 293:fbb8e8a0:test-rados-api-smithi043-81880-80::foo:2, removing 2024-03-21T13:02:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:09 smithi155 ceph-mon[33846]: pgmap v663: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:02:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:09 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:02:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:09 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:09 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:09.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:09 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-79"}]: dispatch 2024-03-21T13:02:09.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:09 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-79"}]: dispatch 2024-03-21T13:02:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:09 smithi043 ceph-mon[30062]: pgmap v663: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:02:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:09 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:02:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:09 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:09 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:09 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-79"}]: dispatch 2024-03-21T13:02:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:09 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-79"}]: dispatch 2024-03-21T13:02:09.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:09 smithi043 ceph-mon[41699]: pgmap v663: 356 pgs: 356 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:02:09.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:09 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:02:09.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:09 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:09.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:09 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:09.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:09 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-79"}]: dispatch 2024-03-21T13:02:09.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:09 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-79"}]: dispatch 2024-03-21T13:02:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:10 smithi155 ceph-mon[33846]: osdmap e437: 8 total, 8 up, 8 in 2024-03-21T13:02:10.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:10 smithi043 ceph-mon[30062]: osdmap e437: 8 total, 8 up, 8 in 2024-03-21T13:02:10.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:10 smithi043 ceph-mon[41699]: osdmap e437: 8 total, 8 up, 8 in 2024-03-21T13:02:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:11 smithi155 ceph-mon[33846]: pgmap v665: 324 pgs: 324 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:02:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:11 smithi155 ceph-mon[33846]: osdmap e438: 8 total, 8 up, 8 in 2024-03-21T13:02:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:11 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:11 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:11 smithi043 ceph-mon[30062]: pgmap v665: 324 pgs: 324 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:02:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:11 smithi043 ceph-mon[30062]: osdmap e438: 8 total, 8 up, 8 in 2024-03-21T13:02:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:11 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:11 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:11.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:11 smithi043 ceph-mon[41699]: pgmap v665: 324 pgs: 324 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:02:11.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:11 smithi043 ceph-mon[41699]: osdmap e438: 8 total, 8 up, 8 in 2024-03-21T13:02:11.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:11 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:11.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:11 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:12 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:02:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:12 smithi155 ceph-mon[33846]: osdmap e439: 8 total, 8 up, 8 in 2024-03-21T13:02:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:12 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:02:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:12 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:12.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:12 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:12 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:02:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:12 smithi043 ceph-mon[30062]: osdmap e439: 8 total, 8 up, 8 in 2024-03-21T13:02:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:12 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:02:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:12 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:12 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:12.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:12 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:02:12.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:12 smithi043 ceph-mon[41699]: osdmap e439: 8 total, 8 up, 8 in 2024-03-21T13:02:12.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:12 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:02:12.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:12 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:12.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:12 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:13 smithi155 ceph-mon[33846]: pgmap v668: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:02:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:13 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:02:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:13 smithi155 ceph-mon[33846]: osdmap e440: 8 total, 8 up, 8 in 2024-03-21T13:02:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:13 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:13.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:13 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:13 smithi043 ceph-mon[30062]: pgmap v668: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:02:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:13 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:02:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:13 smithi043 ceph-mon[30062]: osdmap e440: 8 total, 8 up, 8 in 2024-03-21T13:02:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:13 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:13 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:13.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:13 smithi043 ceph-mon[41699]: pgmap v668: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 635 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:02:13.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:13 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:02:13.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:13 smithi043 ceph-mon[41699]: osdmap e440: 8 total, 8 up, 8 in 2024-03-21T13:02:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:13 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:13.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:13 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:14.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:14 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T13:02:14.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:14 smithi155 ceph-mon[33846]: osdmap e441: 8 total, 8 up, 8 in 2024-03-21T13:02:14.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:14 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:02:14.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:14 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:02:14.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:14 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T13:02:14.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:14 smithi043 ceph-mon[30062]: osdmap e441: 8 total, 8 up, 8 in 2024-03-21T13:02:14.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:14 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:02:14.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:14 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:02:14.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:14 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T13:02:14.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:14 smithi043 ceph-mon[41699]: osdmap e441: 8 total, 8 up, 8 in 2024-03-21T13:02:14.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:14 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:02:14.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:14 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:02:15.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:15 smithi155 ceph-mon[33846]: pgmap v671: 356 pgs: 356 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:02:15.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:15 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:02:15.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:15 smithi155 ceph-mon[33846]: osdmap e442: 8 total, 8 up, 8 in 2024-03-21T13:02:15.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:15 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:15.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:15 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:15.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:15 smithi043 ceph-mon[30062]: pgmap v671: 356 pgs: 356 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:02:15.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:15 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:02:15.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:15 smithi043 ceph-mon[30062]: osdmap e442: 8 total, 8 up, 8 in 2024-03-21T13:02:15.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:15 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:15.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:15 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:15.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:15 smithi043 ceph-mon[41699]: pgmap v671: 356 pgs: 356 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:02:15.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:15 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:02:15.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:15 smithi043 ceph-mon[41699]: osdmap e442: 8 total, 8 up, 8 in 2024-03-21T13:02:15.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:15 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:15.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:15 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:16.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:16 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:02:16.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:16 smithi155 ceph-mon[33846]: osdmap e443: 8 total, 8 up, 8 in 2024-03-21T13:02:16.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:16 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-21T13:02:16.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:16 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-21T13:02:16.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:16 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:02:16.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:16 smithi043 ceph-mon[30062]: osdmap e443: 8 total, 8 up, 8 in 2024-03-21T13:02:16.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:16 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-21T13:02:16.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:16 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-21T13:02:16.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:16 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:02:16.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:16 smithi043 ceph-mon[41699]: osdmap e443: 8 total, 8 up, 8 in 2024-03-21T13:02:16.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:16 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-21T13:02:16.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:16 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-21T13:02:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:17 smithi043 ceph-mon[30062]: pgmap v674: 356 pgs: 356 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:02:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:17 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-03-21T13:02:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:17 smithi043 ceph-mon[30062]: osdmap e444: 8 total, 8 up, 8 in 2024-03-21T13:02:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:17 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-21T13:02:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:17 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-21T13:02:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.10", "id": [6, 4]}]: dispatch 2024-03-21T13:02:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:17 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.15", "id": [6, 4]}]: dispatch 2024-03-21T13:02:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:17 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.10", "id": [6, 4]}]: dispatch 2024-03-21T13:02:17.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:17 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.15", "id": [6, 4]}]: dispatch 2024-03-21T13:02:17.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:17 smithi043 ceph-mon[41699]: pgmap v674: 356 pgs: 356 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:02:17.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:17 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-03-21T13:02:17.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:17 smithi043 ceph-mon[41699]: osdmap e444: 8 total, 8 up, 8 in 2024-03-21T13:02:17.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:17 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-21T13:02:17.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:17 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-21T13:02:17.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.10", "id": [6, 4]}]: dispatch 2024-03-21T13:02:17.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:17 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.15", "id": [6, 4]}]: dispatch 2024-03-21T13:02:17.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:17 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.10", "id": [6, 4]}]: dispatch 2024-03-21T13:02:17.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:17 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.15", "id": [6, 4]}]: dispatch 2024-03-21T13:02:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:17 smithi155 ceph-mon[33846]: pgmap v674: 356 pgs: 356 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:02:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:17 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-03-21T13:02:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:17 smithi155 ceph-mon[33846]: osdmap e444: 8 total, 8 up, 8 in 2024-03-21T13:02:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:17 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-21T13:02:18.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:17 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-21T13:02:18.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.10", "id": [6, 4]}]: dispatch 2024-03-21T13:02:18.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:17 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.15", "id": [6, 4]}]: dispatch 2024-03-21T13:02:18.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:17 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.10", "id": [6, 4]}]: dispatch 2024-03-21T13:02:18.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:17 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.15", "id": [6, 4]}]: dispatch 2024-03-21T13:02:18.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:02:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:02:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:02:18.615 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552539e00 2024-03-21T13:02:18.615 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525395b0 2024-03-21T13:02:18.615 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552539f30 2024-03-21T13:02:18.615 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253a060 2024-03-21T13:02:18.615 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552539930 2024-03-21T13:02:18.615 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552539a80 2024-03-21T13:02:18.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253a640 2024-03-21T13:02:18.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253a190 2024-03-21T13:02:18.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253a2c0 2024-03-21T13:02:18.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253a3f0 2024-03-21T13:02:18.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253a520 2024-03-21T13:02:18.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253ac20 2024-03-21T13:02:18.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253a770 2024-03-21T13:02:18.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253a8c0 2024-03-21T13:02:18.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253a9f0 2024-03-21T13:02:18.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253ad50 2024-03-21T13:02:18.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253b220 2024-03-21T13:02:18.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253b350 2024-03-21T13:02:18.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525384b0 2024-03-21T13:02:18.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525385e0 2024-03-21T13:02:18.616 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552538710 2024-03-21T13:02:18.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253b480 2024-03-21T13:02:18.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253b950 2024-03-21T13:02:18.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253baa0 2024-03-21T13:02:18.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253ae80 2024-03-21T13:02:18.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253afb0 2024-03-21T13:02:18.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253b0e0 2024-03-21T13:02:18.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253bbd0 2024-03-21T13:02:18.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253c0a0 2024-03-21T13:02:18.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253c1d0 2024-03-21T13:02:18.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552539bb0 2024-03-21T13:02:18.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253c320 2024-03-21T13:02:18.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253c470 2024-03-21T13:02:18.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253bd00 2024-03-21T13:02:18.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253be30 2024-03-21T13:02:18.617 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253bf60 2024-03-21T13:02:18.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253c5c0 2024-03-21T13:02:18.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253ca90 2024-03-21T13:02:18.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253cbe0 2024-03-21T13:02:18.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253b5b0 2024-03-21T13:02:18.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253b6e0 2024-03-21T13:02:18.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253b810 2024-03-21T13:02:18.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253cd10 2024-03-21T13:02:18.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253d1e0 2024-03-21T13:02:18.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253d310 2024-03-21T13:02:18.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253c6f0 2024-03-21T13:02:18.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253c820 2024-03-21T13:02:18.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253c950 2024-03-21T13:02:18.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253d460 2024-03-21T13:02:18.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253d930 2024-03-21T13:02:18.618 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253da80 2024-03-21T13:02:18.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253ce20 2024-03-21T13:02:18.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253cf70 2024-03-21T13:02:18.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253d0a0 2024-03-21T13:02:18.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253dbb0 2024-03-21T13:02:18.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253e080 2024-03-21T13:02:18.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253e1b0 2024-03-21T13:02:18.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253d590 2024-03-21T13:02:18.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253d6c0 2024-03-21T13:02:18.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253d7f0 2024-03-21T13:02:18.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253e300 2024-03-21T13:02:18.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253e820 2024-03-21T13:02:18.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253e970 2024-03-21T13:02:18.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253dce0 2024-03-21T13:02:18.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253de10 2024-03-21T13:02:18.619 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253df40 2024-03-21T13:02:18.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253eaa0 2024-03-21T13:02:18.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253ef70 2024-03-21T13:02:18.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253f0c0 2024-03-21T13:02:18.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253e430 2024-03-21T13:02:18.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253e560 2024-03-21T13:02:18.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253e690 2024-03-21T13:02:18.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253f1f0 2024-03-21T13:02:18.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253f6c0 2024-03-21T13:02:18.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253f7f0 2024-03-21T13:02:18.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253ebd0 2024-03-21T13:02:18.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253ed00 2024-03-21T13:02:18.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253ee30 2024-03-21T13:02:18.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253f920 2024-03-21T13:02:18.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253fe40 2024-03-21T13:02:18.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253ff90 2024-03-21T13:02:18.620 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253f320 2024-03-21T13:02:18.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:02:18 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:02:18.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:18 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-03-21T13:02:18.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:18 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.10", "id": [6, 4]}]': finished 2024-03-21T13:02:18.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:18 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.15", "id": [6, 4]}]': finished 2024-03-21T13:02:18.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:18 smithi155 ceph-mon[33846]: osdmap e445: 8 total, 8 up, 8 in 2024-03-21T13:02:18.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:18 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:18.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:18 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:18.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:18 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-03-21T13:02:18.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:18 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.10", "id": [6, 4]}]': finished 2024-03-21T13:02:18.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:18 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.15", "id": [6, 4]}]': finished 2024-03-21T13:02:18.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:18 smithi043 ceph-mon[30062]: osdmap e445: 8 total, 8 up, 8 in 2024-03-21T13:02:18.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:18 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:18.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:18 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:18.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:18 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-03-21T13:02:18.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:18 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.10", "id": [6, 4]}]': finished 2024-03-21T13:02:18.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:18 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "294.15", "id": [6, 4]}]': finished 2024-03-21T13:02:18.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:18 smithi043 ceph-mon[41699]: osdmap e445: 8 total, 8 up, 8 in 2024-03-21T13:02:18.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:18 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:18.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:18 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:19 smithi043 ceph-mon[30062]: pgmap v677: 356 pgs: 356 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:02:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:19 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:02:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:19 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:02:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:19 smithi043 ceph-mon[30062]: osdmap e446: 8 total, 8 up, 8 in 2024-03-21T13:02:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:19 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:19 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:19 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-81"}]: dispatch 2024-03-21T13:02:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:19 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-81"}]: dispatch 2024-03-21T13:02:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:19 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:02:19.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:19 smithi043 ceph-mon[41699]: pgmap v677: 356 pgs: 356 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:02:19.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:19 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:02:19.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:19 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:02:19.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:19 smithi043 ceph-mon[41699]: osdmap e446: 8 total, 8 up, 8 in 2024-03-21T13:02:19.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:19 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:19.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:19 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:19.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:19 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-81"}]: dispatch 2024-03-21T13:02:19.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:19 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-81"}]: dispatch 2024-03-21T13:02:19.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:19 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:02:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:19 smithi155 ceph-mon[33846]: pgmap v677: 356 pgs: 356 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:02:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:19 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:02:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:19 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:02:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:19 smithi155 ceph-mon[33846]: osdmap e446: 8 total, 8 up, 8 in 2024-03-21T13:02:20.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:19 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:20.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:19 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:20.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:19 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-81"}]: dispatch 2024-03-21T13:02:20.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:19 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-81"}]: dispatch 2024-03-21T13:02:20.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:19 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:02:20.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:20 smithi043 ceph-mon[30062]: osdmap e447: 8 total, 8 up, 8 in 2024-03-21T13:02:20.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:20 smithi043 ceph-mon[41699]: osdmap e447: 8 total, 8 up, 8 in 2024-03-21T13:02:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:20 smithi155 ceph-mon[33846]: osdmap e447: 8 total, 8 up, 8 in 2024-03-21T13:02:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:21 smithi043 ceph-mon[30062]: pgmap v680: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 7.5 KiB/s rd, 9.0 KiB/s wr, 28 op/s 2024-03-21T13:02:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:21 smithi043 ceph-mon[30062]: osdmap e448: 8 total, 8 up, 8 in 2024-03-21T13:02:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:21 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:21 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:21.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:21 smithi043 ceph-mon[41699]: pgmap v680: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 7.5 KiB/s rd, 9.0 KiB/s wr, 28 op/s 2024-03-21T13:02:21.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:21 smithi043 ceph-mon[41699]: osdmap e448: 8 total, 8 up, 8 in 2024-03-21T13:02:21.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:21 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:21.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:21 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:22.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:21 smithi155 ceph-mon[33846]: pgmap v680: 324 pgs: 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 7.5 KiB/s rd, 9.0 KiB/s wr, 28 op/s 2024-03-21T13:02:22.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:21 smithi155 ceph-mon[33846]: osdmap e448: 8 total, 8 up, 8 in 2024-03-21T13:02:22.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:21 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:22.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:21 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:23.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:22 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:02:23.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:22 smithi155 ceph-mon[33846]: osdmap e449: 8 total, 8 up, 8 in 2024-03-21T13:02:23.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:22 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:02:23.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:22 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:23.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:22 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:22 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:02:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:22 smithi043 ceph-mon[30062]: osdmap e449: 8 total, 8 up, 8 in 2024-03-21T13:02:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:22 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:02:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:22 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:22 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:23.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:22 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:02:23.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:22 smithi043 ceph-mon[41699]: osdmap e449: 8 total, 8 up, 8 in 2024-03-21T13:02:23.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:22 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:02:23.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:22 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:23.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:22 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:23 smithi155 ceph-mon[33846]: pgmap v683: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 7.5 KiB/s rd, 9.0 KiB/s wr, 28 op/s 2024-03-21T13:02:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:23 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:02:24.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:23 smithi155 ceph-mon[33846]: osdmap e450: 8 total, 8 up, 8 in 2024-03-21T13:02:24.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:23 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:24.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:23 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:24.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:23 smithi043 ceph-mon[30062]: pgmap v683: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 7.5 KiB/s rd, 9.0 KiB/s wr, 28 op/s 2024-03-21T13:02:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:23 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:02:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:23 smithi043 ceph-mon[30062]: osdmap e450: 8 total, 8 up, 8 in 2024-03-21T13:02:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:23 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:23 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:24.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:23 smithi043 ceph-mon[41699]: pgmap v683: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 644 MiB used, 715 GiB / 715 GiB avail; 7.5 KiB/s rd, 9.0 KiB/s wr, 28 op/s 2024-03-21T13:02:24.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:23 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:02:24.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:23 smithi043 ceph-mon[41699]: osdmap e450: 8 total, 8 up, 8 in 2024-03-21T13:02:24.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:23 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:24.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:23 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:25.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:24 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T13:02:25.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:24 smithi155 ceph-mon[33846]: osdmap e451: 8 total, 8 up, 8 in 2024-03-21T13:02:25.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:24 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:02:25.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:24 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:02:25.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:24 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:02:25.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:24 smithi155 ceph-mon[33846]: osdmap e452: 8 total, 8 up, 8 in 2024-03-21T13:02:25.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:24 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T13:02:25.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:24 smithi043 ceph-mon[30062]: osdmap e451: 8 total, 8 up, 8 in 2024-03-21T13:02:25.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:24 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:02:25.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:24 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:02:25.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:24 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:02:25.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:24 smithi043 ceph-mon[30062]: osdmap e452: 8 total, 8 up, 8 in 2024-03-21T13:02:25.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:24 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T13:02:25.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:24 smithi043 ceph-mon[41699]: osdmap e451: 8 total, 8 up, 8 in 2024-03-21T13:02:25.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:24 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:02:25.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:24 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:02:25.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:24 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:02:25.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:24 smithi043 ceph-mon[41699]: osdmap e452: 8 total, 8 up, 8 in 2024-03-21T13:02:26.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:25 smithi155 ceph-mon[33846]: pgmap v686: 356 pgs: 356 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-03-21T13:02:26.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:25 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:26.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:25 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:26.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:25 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:02:26.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:25 smithi155 ceph-mon[33846]: osdmap e453: 8 total, 8 up, 8 in 2024-03-21T13:02:26.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:25 smithi043 ceph-mon[30062]: pgmap v686: 356 pgs: 356 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-03-21T13:02:26.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:25 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:26.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:25 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:26.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:25 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:02:26.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:25 smithi043 ceph-mon[30062]: osdmap e453: 8 total, 8 up, 8 in 2024-03-21T13:02:26.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:25 smithi043 ceph-mon[41699]: pgmap v686: 356 pgs: 356 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-03-21T13:02:26.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:25 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:26.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:25 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:26.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:25 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:02:26.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:25 smithi043 ceph-mon[41699]: osdmap e453: 8 total, 8 up, 8 in 2024-03-21T13:02:28.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:27 smithi043 ceph-mon[30062]: pgmap v689: 356 pgs: 356 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-03-21T13:02:28.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:27 smithi043 ceph-mon[30062]: osdmap e454: 8 total, 8 up, 8 in 2024-03-21T13:02:28.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:27 smithi043 ceph-mon[41699]: pgmap v689: 356 pgs: 356 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-03-21T13:02:28.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:27 smithi043 ceph-mon[41699]: osdmap e454: 8 total, 8 up, 8 in 2024-03-21T13:02:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:27 smithi155 ceph-mon[33846]: pgmap v689: 356 pgs: 356 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 3.7 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-03-21T13:02:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:27 smithi155 ceph-mon[33846]: osdmap e454: 8 total, 8 up, 8 in 2024-03-21T13:02:28.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:02:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:02:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:02:28.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:02:28 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:02:28.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:28 smithi155 ceph-mon[33846]: osdmap e455: 8 total, 8 up, 8 in 2024-03-21T13:02:28.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:28.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:28 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:28.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-83"}]: dispatch 2024-03-21T13:02:28.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:28 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-83"}]: dispatch 2024-03-21T13:02:28.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:28 smithi155 ceph-mon[33846]: pgmap v692: 356 pgs: 356 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:02:28.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:28 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:02:29.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:28 smithi043 ceph-mon[41699]: osdmap e455: 8 total, 8 up, 8 in 2024-03-21T13:02:29.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:29.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:28 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:29.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-83"}]: dispatch 2024-03-21T13:02:29.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:28 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-83"}]: dispatch 2024-03-21T13:02:29.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:28 smithi043 ceph-mon[41699]: pgmap v692: 356 pgs: 356 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:02:29.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:28 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:02:29.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:28 smithi043 ceph-mon[30062]: osdmap e455: 8 total, 8 up, 8 in 2024-03-21T13:02:29.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:29.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:28 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:29.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-83"}]: dispatch 2024-03-21T13:02:29.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:28 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-83"}]: dispatch 2024-03-21T13:02:29.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:28 smithi043 ceph-mon[30062]: pgmap v692: 356 pgs: 356 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:02:29.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:28 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:02:30.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:29 smithi155 ceph-mon[33846]: osdmap e456: 8 total, 8 up, 8 in 2024-03-21T13:02:30.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:29 smithi043 ceph-mon[30062]: osdmap e456: 8 total, 8 up, 8 in 2024-03-21T13:02:30.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:29 smithi043 ceph-mon[41699]: osdmap e456: 8 total, 8 up, 8 in 2024-03-21T13:02:31.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:30 smithi155 ceph-mon[33846]: osdmap e457: 8 total, 8 up, 8 in 2024-03-21T13:02:31.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:30 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:31.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:30 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:31.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:30 smithi155 ceph-mon[33846]: pgmap v695: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-21T13:02:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:30 smithi043 ceph-mon[30062]: osdmap e457: 8 total, 8 up, 8 in 2024-03-21T13:02:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:30 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:30 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:30 smithi043 ceph-mon[30062]: pgmap v695: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-21T13:02:31.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:30 smithi043 ceph-mon[41699]: osdmap e457: 8 total, 8 up, 8 in 2024-03-21T13:02:31.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:30 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:31.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:30 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:31.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:30 smithi043 ceph-mon[41699]: pgmap v695: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-21T13:02:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:31 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:02:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:31 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:02:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:31 smithi155 ceph-mon[33846]: osdmap e458: 8 total, 8 up, 8 in 2024-03-21T13:02:32.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:31 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:02:32.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:31 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:32.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:31 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:31 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:02:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:31 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:02:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:31 smithi043 ceph-mon[30062]: osdmap e458: 8 total, 8 up, 8 in 2024-03-21T13:02:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:31 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:02:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:31 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:31 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:31 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:02:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:31 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:02:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:31 smithi043 ceph-mon[41699]: osdmap e458: 8 total, 8 up, 8 in 2024-03-21T13:02:32.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:31 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:02:32.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:31 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:32.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:31 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:33.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:32 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:02:33.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:32 smithi155 ceph-mon[33846]: osdmap e459: 8 total, 8 up, 8 in 2024-03-21T13:02:33.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:32 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:33.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:32 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:33.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:32 smithi155 ceph-mon[33846]: pgmap v698: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-21T13:02:33.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:32 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:02:33.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:32 smithi043 ceph-mon[30062]: osdmap e459: 8 total, 8 up, 8 in 2024-03-21T13:02:33.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:32 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:33.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:32 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:33.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:32 smithi043 ceph-mon[30062]: pgmap v698: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-21T13:02:33.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:32 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:02:33.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:32 smithi043 ceph-mon[41699]: osdmap e459: 8 total, 8 up, 8 in 2024-03-21T13:02:33.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:32 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:33.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:32 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:33.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:32 smithi043 ceph-mon[41699]: pgmap v698: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-21T13:02:34.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:33 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T13:02:34.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:33 smithi155 ceph-mon[33846]: osdmap e460: 8 total, 8 up, 8 in 2024-03-21T13:02:34.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:33 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:02:34.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:33 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:02:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:33 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T13:02:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:33 smithi043 ceph-mon[41699]: osdmap e460: 8 total, 8 up, 8 in 2024-03-21T13:02:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:33 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:02:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:33 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:02:34.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:33 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_tier","val": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T13:02:34.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:33 smithi043 ceph-mon[30062]: osdmap e460: 8 total, 8 up, 8 in 2024-03-21T13:02:34.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:33 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:02:34.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:33 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:02:35.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:34 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:02:35.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:34 smithi155 ceph-mon[33846]: osdmap e461: 8 total, 8 up, 8 in 2024-03-21T13:02:35.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:34 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:35.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:34 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:35.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:34 smithi155 ceph-mon[33846]: pgmap v701: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:02:35.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:34 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:02:35.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:34 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:02:35.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:34 smithi155 ceph-mon[33846]: osdmap e462: 8 total, 8 up, 8 in 2024-03-21T13:02:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:34 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:02:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:34 smithi043 ceph-mon[30062]: osdmap e461: 8 total, 8 up, 8 in 2024-03-21T13:02:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:34 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:34 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:34 smithi043 ceph-mon[30062]: pgmap v701: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:02:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:34 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:02:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:34 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:02:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:34 smithi043 ceph-mon[30062]: osdmap e462: 8 total, 8 up, 8 in 2024-03-21T13:02:35.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:34 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:02:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:34 smithi043 ceph-mon[41699]: osdmap e461: 8 total, 8 up, 8 in 2024-03-21T13:02:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:34 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:34 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:34 smithi043 ceph-mon[41699]: pgmap v701: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:02:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:34 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:02:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:34 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:02:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:34 smithi043 ceph-mon[41699]: osdmap e462: 8 total, 8 up, 8 in 2024-03-21T13:02:37.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:36 smithi155 ceph-mon[33846]: osdmap e463: 8 total, 8 up, 8 in 2024-03-21T13:02:37.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:36 smithi155 ceph-mon[33846]: pgmap v704: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:02:37.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:36 smithi043 ceph-mon[30062]: osdmap e463: 8 total, 8 up, 8 in 2024-03-21T13:02:37.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:36 smithi043 ceph-mon[30062]: pgmap v704: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:02:37.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:36 smithi043 ceph-mon[41699]: osdmap e463: 8 total, 8 up, 8 in 2024-03-21T13:02:37.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:36 smithi043 ceph-mon[41699]: pgmap v704: 356 pgs: 356 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:02:38.135 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:02:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:02:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:02:38.136 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:37 smithi043 ceph-mon[30062]: osdmap e464: 8 total, 8 up, 8 in 2024-03-21T13:02:38.136 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:37 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:38.136 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:37 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:38.136 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:37 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-85"}]: dispatch 2024-03-21T13:02:38.136 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:37 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-85"}]: dispatch 2024-03-21T13:02:38.136 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:37 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:02:38.136 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:37 smithi043 ceph-mon[41699]: osdmap e464: 8 total, 8 up, 8 in 2024-03-21T13:02:38.136 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:37 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:38.136 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:37 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:38.137 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:37 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-85"}]: dispatch 2024-03-21T13:02:38.137 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:37 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-85"}]: dispatch 2024-03-21T13:02:38.137 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:37 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:02:38.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:37 smithi155 ceph-mon[33846]: osdmap e464: 8 total, 8 up, 8 in 2024-03-21T13:02:38.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:37 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:38.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:37 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:38.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:37 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-85"}]: dispatch 2024-03-21T13:02:38.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:37 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-85"}]: dispatch 2024-03-21T13:02:38.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:37 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:02:38.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:38 smithi155 ceph-mon[33846]: osdmap e465: 8 total, 8 up, 8 in 2024-03-21T13:02:38.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:38 smithi155 ceph-mon[33846]: pgmap v707: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:02:38.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:38 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:02:38.958 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:02:38 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:02:39.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:38 smithi043 ceph-mon[30062]: osdmap e465: 8 total, 8 up, 8 in 2024-03-21T13:02:39.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:38 smithi043 ceph-mon[30062]: pgmap v707: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:02:39.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:38 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:02:39.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:38 smithi043 ceph-mon[41699]: osdmap e465: 8 total, 8 up, 8 in 2024-03-21T13:02:39.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:38 smithi043 ceph-mon[41699]: pgmap v707: 324 pgs: 324 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:02:39.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:38 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:02:39.848 INFO:tasks.workunit.client.0.smithi043.stdout: OK ] LibRadosTwoPoolsPP.ProxyRead (18073 ms) 2024-03-21T13:02:39.848 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-03-21T13:02:39.848 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (22559 ms) 2024-03-21T13:02:39.848 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-03-21T13:02:39.848 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3012 ms) 2024-03-21T13:02:39.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-03-21T13:02:39.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:02:39.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3031 ms) 2024-03-21T13:02:39.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-03-21T13:02:39.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3016 ms) 2024-03-21T13:02:39.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-03-21T13:02:39.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:02:39.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3021 ms) 2024-03-21T13:02:39.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-03-21T13:02:39.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:02:39.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4037 ms) 2024-03-21T13:02:39.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-03-21T13:02:39.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:02:39.849 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (38274 ms) 2024-03-21T13:02:39.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-03-21T13:02:39.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:02:39.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (17069 ms) 2024-03-21T13:02:39.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-03-21T13:02:39.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:02:39.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4030 ms) 2024-03-21T13:02:39.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-03-21T13:02:39.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:02:39.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3012 ms) 2024-03-21T13:02:39.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-03-21T13:02:39.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:02:39.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (24362 ms) 2024-03-21T13:02:39.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-03-21T13:02:39.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:02:39.850 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (15057 ms) 2024-03-21T13:02:39.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-03-21T13:02:39.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:02:39.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5022 ms) 2024-03-21T13:02:39.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-03-21T13:02:39.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:02:39.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (4010 ms) 2024-03-21T13:02:39.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-03-21T13:02:39.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:02:39.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: waiting for scrubs... 2024-03-21T13:02:39.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: done waiting 2024-03-21T13:02:39.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (24428 ms) 2024-03-21T13:02:39.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-03-21T13:02:39.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:02:39.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10075 ms) 2024-03-21T13:02:39.851 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-03-21T13:02:39.852 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:02:39.852 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9044 ms) 2024-03-21T13:02:39.852 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-03-21T13:02:39.852 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:02:39.852 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9054 ms) 2024-03-21T13:02:39.852 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-03-21T13:02:40.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:39 smithi155 ceph-mon[33846]: osdmap e466: 8 total, 8 up, 8 in 2024-03-21T13:02:40.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:39 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:40.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:39 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:40.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:39 smithi043 ceph-mon[30062]: osdmap e466: 8 total, 8 up, 8 in 2024-03-21T13:02:40.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:39 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:40.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:39 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:39 smithi043 ceph-mon[41699]: osdmap e466: 8 total, 8 up, 8 in 2024-03-21T13:02:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:39 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:39 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:40 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:02:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:40 smithi155 ceph-mon[33846]: osdmap e467: 8 total, 8 up, 8 in 2024-03-21T13:02:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:40 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:02:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:40 smithi155 ceph-mon[33846]: pgmap v710: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-03-21T13:02:41.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:40 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:02:41.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:40 smithi043 ceph-mon[30062]: osdmap e467: 8 total, 8 up, 8 in 2024-03-21T13:02:41.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:40 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:02:41.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:40 smithi043 ceph-mon[30062]: pgmap v710: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-03-21T13:02:41.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:40 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:02:41.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:40 smithi043 ceph-mon[41699]: osdmap e467: 8 total, 8 up, 8 in 2024-03-21T13:02:41.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:40 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:02:41.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:40 smithi043 ceph-mon[41699]: pgmap v710: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-03-21T13:02:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:41 smithi155 ceph-mon[33846]: osdmap e468: 8 total, 8 up, 8 in 2024-03-21T13:02:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:41 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:42.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:41 smithi043 ceph-mon[30062]: osdmap e468: 8 total, 8 up, 8 in 2024-03-21T13:02:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:41 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:41 smithi043 ceph-mon[41699]: osdmap e468: 8 total, 8 up, 8 in 2024-03-21T13:02:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:41 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:42 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:02:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:42 smithi155 ceph-mon[33846]: osdmap e469: 8 total, 8 up, 8 in 2024-03-21T13:02:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_tier","val": "test-rados-api-smithi043-81880-89-test-flush"}]: dispatch 2024-03-21T13:02:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:42 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_tier","val": "test-rados-api-smithi043-81880-89-test-flush"}]: dispatch 2024-03-21T13:02:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:42 smithi155 ceph-mon[33846]: pgmap v713: 388 pgs: 64 unknown, 324 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-03-21T13:02:43.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:42 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_tier","val": "test-rados-api-smithi043-81880-89-test-flush"}]': finished 2024-03-21T13:02:43.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:42 smithi155 ceph-mon[33846]: osdmap e470: 8 total, 8 up, 8 in 2024-03-21T13:02:43.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:02:43.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:42 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:02:43.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:42 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:02:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:42 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:02:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:42 smithi043 ceph-mon[30062]: osdmap e469: 8 total, 8 up, 8 in 2024-03-21T13:02:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_tier","val": "test-rados-api-smithi043-81880-89-test-flush"}]: dispatch 2024-03-21T13:02:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:42 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_tier","val": "test-rados-api-smithi043-81880-89-test-flush"}]: dispatch 2024-03-21T13:02:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:42 smithi043 ceph-mon[30062]: pgmap v713: 388 pgs: 64 unknown, 324 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-03-21T13:02:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:42 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_tier","val": "test-rados-api-smithi043-81880-89-test-flush"}]': finished 2024-03-21T13:02:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:42 smithi043 ceph-mon[30062]: osdmap e470: 8 total, 8 up, 8 in 2024-03-21T13:02:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:02:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:42 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:02:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:42 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:02:43.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:42 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:02:43.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:42 smithi043 ceph-mon[41699]: osdmap e469: 8 total, 8 up, 8 in 2024-03-21T13:02:43.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_tier","val": "test-rados-api-smithi043-81880-89-test-flush"}]: dispatch 2024-03-21T13:02:43.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:42 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_tier","val": "test-rados-api-smithi043-81880-89-test-flush"}]: dispatch 2024-03-21T13:02:43.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:42 smithi043 ceph-mon[41699]: pgmap v713: 388 pgs: 64 unknown, 324 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-03-21T13:02:43.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:42 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_tier","val": "test-rados-api-smithi043-81880-89-test-flush"}]': finished 2024-03-21T13:02:43.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:42 smithi043 ceph-mon[41699]: osdmap e470: 8 total, 8 up, 8 in 2024-03-21T13:02:43.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:02:43.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:42 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:02:43.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:42 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:02:45.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:44 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:02:45.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:44 smithi155 ceph-mon[33846]: osdmap e471: 8 total, 8 up, 8 in 2024-03-21T13:02:45.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:44 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:45.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:44 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:45.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:44 smithi155 ceph-mon[33846]: pgmap v716: 388 pgs: 388 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:02:45.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:44 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:02:45.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:44 smithi043 ceph-mon[30062]: osdmap e471: 8 total, 8 up, 8 in 2024-03-21T13:02:45.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:44 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:45.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:44 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:45.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:44 smithi043 ceph-mon[30062]: pgmap v716: 388 pgs: 388 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:02:45.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:44 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:02:45.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:44 smithi043 ceph-mon[41699]: osdmap e471: 8 total, 8 up, 8 in 2024-03-21T13:02:45.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:44 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:45.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:44 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:02:45.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:44 smithi043 ceph-mon[41699]: pgmap v716: 388 pgs: 388 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:02:46.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:45 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:02:46.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:45 smithi155 ceph-mon[33846]: osdmap e472: 8 total, 8 up, 8 in 2024-03-21T13:02:46.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:45 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:02:46.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:45 smithi043 ceph-mon[30062]: osdmap e472: 8 total, 8 up, 8 in 2024-03-21T13:02:46.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:45 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:02:46.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:45 smithi043 ceph-mon[41699]: osdmap e472: 8 total, 8 up, 8 in 2024-03-21T13:02:47.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:46 smithi155 ceph-mon[33846]: osdmap e473: 8 total, 8 up, 8 in 2024-03-21T13:02:47.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:47.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:46 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:47.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-87"}]: dispatch 2024-03-21T13:02:47.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:46 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-87"}]: dispatch 2024-03-21T13:02:47.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:46 smithi155 ceph-mon[33846]: pgmap v719: 356 pgs: 356 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:02:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:46 smithi043 ceph-mon[30062]: osdmap e473: 8 total, 8 up, 8 in 2024-03-21T13:02:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:46 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-87"}]: dispatch 2024-03-21T13:02:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:46 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-87"}]: dispatch 2024-03-21T13:02:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:46 smithi043 ceph-mon[30062]: pgmap v719: 356 pgs: 356 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:02:47.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:46 smithi043 ceph-mon[41699]: osdmap e473: 8 total, 8 up, 8 in 2024-03-21T13:02:47.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:47.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:46 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:47.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-87"}]: dispatch 2024-03-21T13:02:47.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:46 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-87"}]: dispatch 2024-03-21T13:02:47.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:46 smithi043 ceph-mon[41699]: pgmap v719: 356 pgs: 356 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:02:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:47 smithi155 ceph-mon[33846]: osdmap e474: 8 total, 8 up, 8 in 2024-03-21T13:02:48.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:47 smithi043 ceph-mon[30062]: osdmap e474: 8 total, 8 up, 8 in 2024-03-21T13:02:48.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:02:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:02:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:02:48.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:47 smithi043 ceph-mon[41699]: osdmap e474: 8 total, 8 up, 8 in 2024-03-21T13:02:48.878 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:02:48 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:02:49.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:48 smithi155 ceph-mon[33846]: osdmap e475: 8 total, 8 up, 8 in 2024-03-21T13:02:49.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:48 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:49.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:48 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:49.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:48 smithi155 ceph-mon[33846]: pgmap v722: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:02:49.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:48 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:02:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:48 smithi043 ceph-mon[30062]: osdmap e475: 8 total, 8 up, 8 in 2024-03-21T13:02:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:48 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:48 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:48 smithi043 ceph-mon[30062]: pgmap v722: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:02:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:48 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:02:49.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:48 smithi043 ceph-mon[41699]: osdmap e475: 8 total, 8 up, 8 in 2024-03-21T13:02:49.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:48 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:49.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:48 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:49.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:48 smithi043 ceph-mon[41699]: pgmap v722: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:02:49.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:48 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:02:50.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:49 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:02:50.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:49 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:02:50.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:49 smithi155 ceph-mon[33846]: osdmap e476: 8 total, 8 up, 8 in 2024-03-21T13:02:50.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:49 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:02:50.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:49 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:50.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:49 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:50.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:49 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:02:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:49 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:02:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:49 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:02:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:49 smithi043 ceph-mon[30062]: osdmap e476: 8 total, 8 up, 8 in 2024-03-21T13:02:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:49 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:02:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:49 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:49 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:49 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:02:50.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:49 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:02:50.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:49 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:02:50.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:49 smithi043 ceph-mon[41699]: osdmap e476: 8 total, 8 up, 8 in 2024-03-21T13:02:50.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:49 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:02:50.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:49 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:50.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:49 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:02:50.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:49 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:02:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:50 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:02:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:50 smithi155 ceph-mon[33846]: osdmap e477: 8 total, 8 up, 8 in 2024-03-21T13:02:51.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:50 smithi155 ceph-mon[33846]: pgmap v725: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:02:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:50 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:02:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:50 smithi043 ceph-mon[30062]: osdmap e477: 8 total, 8 up, 8 in 2024-03-21T13:02:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:50 smithi043 ceph-mon[30062]: pgmap v725: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:02:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:50 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:02:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:50 smithi043 ceph-mon[41699]: osdmap e477: 8 total, 8 up, 8 in 2024-03-21T13:02:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:50 smithi043 ceph-mon[41699]: pgmap v725: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:02:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:51 smithi155 ceph-mon[33846]: osdmap e478: 8 total, 8 up, 8 in 2024-03-21T13:02:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:51 smithi043 ceph-mon[30062]: osdmap e478: 8 total, 8 up, 8 in 2024-03-21T13:02:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:51 smithi043 ceph-mon[41699]: osdmap e478: 8 total, 8 up, 8 in 2024-03-21T13:02:53.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:52 smithi155 ceph-mon[33846]: osdmap e479: 8 total, 8 up, 8 in 2024-03-21T13:02:53.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:53.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:52 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:53.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-90"}]: dispatch 2024-03-21T13:02:53.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:52 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-90"}]: dispatch 2024-03-21T13:02:53.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:52 smithi155 ceph-mon[33846]: pgmap v728: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:02:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:52 smithi043 ceph-mon[30062]: osdmap e479: 8 total, 8 up, 8 in 2024-03-21T13:02:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:52 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-90"}]: dispatch 2024-03-21T13:02:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:52 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-90"}]: dispatch 2024-03-21T13:02:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:52 smithi043 ceph-mon[30062]: pgmap v728: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:02:53.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:52 smithi043 ceph-mon[41699]: osdmap e479: 8 total, 8 up, 8 in 2024-03-21T13:02:53.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:53.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:52 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:53.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-90"}]: dispatch 2024-03-21T13:02:53.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:52 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-90"}]: dispatch 2024-03-21T13:02:53.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:52 smithi043 ceph-mon[41699]: pgmap v728: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:02:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:53 smithi155 ceph-mon[33846]: osdmap e480: 8 total, 8 up, 8 in 2024-03-21T13:02:54.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:53 smithi043 ceph-mon[30062]: osdmap e480: 8 total, 8 up, 8 in 2024-03-21T13:02:54.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:53 smithi043 ceph-mon[41699]: osdmap e480: 8 total, 8 up, 8 in 2024-03-21T13:02:55.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:54 smithi155 ceph-mon[33846]: osdmap e481: 8 total, 8 up, 8 in 2024-03-21T13:02:55.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:54 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:55.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:54 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:55.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:54 smithi155 ceph-mon[33846]: pgmap v731: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-21T13:02:55.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:54 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:02:55.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:54 smithi155 ceph-mon[33846]: osdmap e482: 8 total, 8 up, 8 in 2024-03-21T13:02:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:54 smithi043 ceph-mon[30062]: osdmap e481: 8 total, 8 up, 8 in 2024-03-21T13:02:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:54 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:54 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:54 smithi043 ceph-mon[30062]: pgmap v731: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-21T13:02:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:54 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:02:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:54 smithi043 ceph-mon[30062]: osdmap e482: 8 total, 8 up, 8 in 2024-03-21T13:02:55.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:54 smithi043 ceph-mon[41699]: osdmap e481: 8 total, 8 up, 8 in 2024-03-21T13:02:55.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:54 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:55.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:54 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:02:55.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:54 smithi043 ceph-mon[41699]: pgmap v731: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-21T13:02:55.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:54 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:02:55.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:54 smithi043 ceph-mon[41699]: osdmap e482: 8 total, 8 up, 8 in 2024-03-21T13:02:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:55 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:02:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:55 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:02:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:55 smithi155 ceph-mon[33846]: osdmap e483: 8 total, 8 up, 8 in 2024-03-21T13:02:56.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:55 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:02:56.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:55 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:02:56.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:55 smithi043 ceph-mon[30062]: osdmap e483: 8 total, 8 up, 8 in 2024-03-21T13:02:56.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:55 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:02:56.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:55 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:02:56.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:55 smithi043 ceph-mon[41699]: osdmap e483: 8 total, 8 up, 8 in 2024-03-21T13:02:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:56 smithi155 ceph-mon[33846]: pgmap v734: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-21T13:02:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:56 smithi155 ceph-mon[33846]: osdmap e484: 8 total, 8 up, 8 in 2024-03-21T13:02:57.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:56 smithi043 ceph-mon[30062]: pgmap v734: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-21T13:02:57.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:56 smithi043 ceph-mon[30062]: osdmap e484: 8 total, 8 up, 8 in 2024-03-21T13:02:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:56 smithi043 ceph-mon[41699]: pgmap v734: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-21T13:02:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:56 smithi043 ceph-mon[41699]: osdmap e484: 8 total, 8 up, 8 in 2024-03-21T13:02:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:57 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:57 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:57 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-92"}]: dispatch 2024-03-21T13:02:58.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:57 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-92"}]: dispatch 2024-03-21T13:02:58.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:02:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:02:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:02:58.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:57 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:58.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:57 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:58.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:57 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-92"}]: dispatch 2024-03-21T13:02:58.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:57 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-92"}]: dispatch 2024-03-21T13:02:58.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:57 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:58.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:57 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:02:58.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:57 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-92"}]: dispatch 2024-03-21T13:02:58.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:57 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-92"}]: dispatch 2024-03-21T13:02:58.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:02:58 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:02:59.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:58 smithi043 ceph-mon[30062]: osdmap e485: 8 total, 8 up, 8 in 2024-03-21T13:02:59.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:58 smithi043 ceph-mon[30062]: pgmap v737: 324 pgs: 324 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:02:59.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:58 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:02:59.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:58 smithi043 ceph-mon[41699]: osdmap e485: 8 total, 8 up, 8 in 2024-03-21T13:02:59.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:58 smithi043 ceph-mon[41699]: pgmap v737: 324 pgs: 324 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:02:59.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:58 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:02:59.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:58 smithi155 ceph-mon[33846]: osdmap e485: 8 total, 8 up, 8 in 2024-03-21T13:02:59.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:58 smithi155 ceph-mon[33846]: pgmap v737: 324 pgs: 324 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:02:59.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:58 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:03:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:59 smithi043 ceph-mon[30062]: osdmap e486: 8 total, 8 up, 8 in 2024-03-21T13:03:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:59 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:03:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:59 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:03:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:02:59 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:03:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:59 smithi043 ceph-mon[41699]: osdmap e486: 8 total, 8 up, 8 in 2024-03-21T13:03:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:59 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:03:00.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:59 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:03:00.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:02:59 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:03:00.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:59 smithi155 ceph-mon[33846]: osdmap e486: 8 total, 8 up, 8 in 2024-03-21T13:03:00.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:59 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:03:00.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:59 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:03:00.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:02:59 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:03:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:03:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: osdmap e487: 8 total, 8 up, 8 in 2024-03-21T13:03:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:03:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: pgmap v740: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:03:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: Adjusting osd_memory_target on smithi043 to 3249M 2024-03-21T13:03:01.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: Adjusting osd_memory_target on smithi155 to 3248M 2024-03-21T13:03:01.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:03:01.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:03:01.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:03:01.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:03:01.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:03:01.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[30062]: osdmap e488: 8 total, 8 up, 8 in 2024-03-21T13:03:01.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:03:01.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: osdmap e487: 8 total, 8 up, 8 in 2024-03-21T13:03:01.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:03:01.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: pgmap v740: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:03:01.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: Adjusting osd_memory_target on smithi043 to 3249M 2024-03-21T13:03:01.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: Adjusting osd_memory_target on smithi155 to 3248M 2024-03-21T13:03:01.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:03:01.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:03:01.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:03:01.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:03:01.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:03:01.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:00 smithi043 ceph-mon[41699]: osdmap e488: 8 total, 8 up, 8 in 2024-03-21T13:03:01.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:03:01.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: osdmap e487: 8 total, 8 up, 8 in 2024-03-21T13:03:01.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:03:01.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: pgmap v740: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:03:01.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T13:03:01.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: Adjusting osd_memory_target on smithi043 to 3249M 2024-03-21T13:03:01.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: Adjusting osd_memory_target on smithi155 to 3248M 2024-03-21T13:03:01.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:03:01.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:03:01.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:03:01.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:03:01.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:03:01.460 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:00 smithi155 ceph-mon[33846]: osdmap e488: 8 total, 8 up, 8 in 2024-03-21T13:03:03.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:02 smithi043 ceph-mon[30062]: osdmap e489: 8 total, 8 up, 8 in 2024-03-21T13:03:03.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:02 smithi043 ceph-mon[30062]: pgmap v743: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:03:03.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:02 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:03:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:02 smithi043 ceph-mon[41699]: osdmap e489: 8 total, 8 up, 8 in 2024-03-21T13:03:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:02 smithi043 ceph-mon[41699]: pgmap v743: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:03:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:02 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:03:03.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:02 smithi155 ceph-mon[33846]: osdmap e489: 8 total, 8 up, 8 in 2024-03-21T13:03:03.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:02 smithi155 ceph-mon[33846]: pgmap v743: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-21T13:03:03.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:02 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:03:05.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:05 smithi155 ceph-mon[33846]: pgmap v744: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-03-21T13:03:05.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:05 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:03:05.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:05 smithi043 ceph-mon[30062]: pgmap v744: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-03-21T13:03:05.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:05 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:03:05.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:05 smithi043 ceph-mon[41699]: pgmap v744: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-03-21T13:03:05.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:05 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:03:07.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:07 smithi155 ceph-mon[33846]: pgmap v745: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-03-21T13:03:07.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:07 smithi043 ceph-mon[30062]: pgmap v745: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-03-21T13:03:07.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:07 smithi043 ceph-mon[41699]: pgmap v745: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-03-21T13:03:08.384 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:03:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:03:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:03:08.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:03:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:03:09.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:09 smithi155 ceph-mon[33846]: pgmap v746: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-21T13:03:09.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:09 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:03:09.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:09 smithi043 ceph-mon[30062]: pgmap v746: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-21T13:03:09.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:09 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:03:09.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:09 smithi043 ceph-mon[41699]: pgmap v746: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-21T13:03:09.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:09 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:03:11.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:11 smithi155 ceph-mon[33846]: pgmap v747: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 898 B/s wr, 3 op/s 2024-03-21T13:03:11.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:11 smithi043 ceph-mon[30062]: pgmap v747: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 898 B/s wr, 3 op/s 2024-03-21T13:03:11.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:11 smithi043 ceph-mon[41699]: pgmap v747: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 898 B/s wr, 3 op/s 2024-03-21T13:03:13.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:13 smithi155 ceph-mon[33846]: pgmap v748: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 810 B/s wr, 3 op/s 2024-03-21T13:03:13.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:13 smithi155 ceph-mon[33846]: osdmap e490: 8 total, 8 up, 8 in 2024-03-21T13:03:13.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:13 smithi043 ceph-mon[30062]: pgmap v748: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 810 B/s wr, 3 op/s 2024-03-21T13:03:13.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:13 smithi043 ceph-mon[30062]: osdmap e490: 8 total, 8 up, 8 in 2024-03-21T13:03:13.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:13 smithi043 ceph-mon[41699]: pgmap v748: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 810 B/s wr, 3 op/s 2024-03-21T13:03:13.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:13 smithi043 ceph-mon[41699]: osdmap e490: 8 total, 8 up, 8 in 2024-03-21T13:03:15.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:15 smithi155 ceph-mon[33846]: pgmap v750: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:03:15.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:15 smithi043 ceph-mon[30062]: pgmap v750: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:03:15.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:15 smithi043 ceph-mon[41699]: pgmap v750: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:03:17.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:17 smithi155 ceph-mon[33846]: pgmap v751: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:03:17.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 13:03:17 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=cleanup t=2024-03-21T13:03:17.148588299Z level=info msg="Completed cleanup jobs" duration=2.41983ms 2024-03-21T13:03:17.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 13:03:17 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=sqlstore.transactions t=2024-03-21T13:03:17.156813009Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-03-21T13:03:17.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:17 smithi043 ceph-mon[30062]: pgmap v751: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:03:17.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:17 smithi043 ceph-mon[41699]: pgmap v751: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:03:18.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:03:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:03:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:03:18.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:18 smithi155 ceph-mon[33846]: osdmap e491: 8 total, 8 up, 8 in 2024-03-21T13:03:18.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:03:18 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:03:18.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:18 smithi043 ceph-mon[30062]: osdmap e491: 8 total, 8 up, 8 in 2024-03-21T13:03:18.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:18 smithi043 ceph-mon[41699]: osdmap e491: 8 total, 8 up, 8 in 2024-03-21T13:03:19.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:19 smithi155 ceph-mon[33846]: pgmap v753: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:03:19.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:19 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:03:19.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:19 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:03:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:19 smithi043 ceph-mon[30062]: pgmap v753: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:03:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:19 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:03:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:19 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:03:19.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:19 smithi043 ceph-mon[41699]: pgmap v753: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:03:19.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:19 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:03:19.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:19 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:03:21.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:21 smithi155 ceph-mon[33846]: pgmap v754: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:03:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:21 smithi043 ceph-mon[30062]: pgmap v754: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:03:21.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:21 smithi043 ceph-mon[41699]: pgmap v754: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:03:22.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:22 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:03:22.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:22 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:03:22.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:22 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-94"}]: dispatch 2024-03-21T13:03:22.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:22 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-94"}]: dispatch 2024-03-21T13:03:22.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:22 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:03:22.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:22 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:03:22.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:22 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-94"}]: dispatch 2024-03-21T13:03:22.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:22 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-94"}]: dispatch 2024-03-21T13:03:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:22 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:03:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:22 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:03:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:22 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-94"}]: dispatch 2024-03-21T13:03:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:22 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-94"}]: dispatch 2024-03-21T13:03:23.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:23 smithi155 ceph-mon[33846]: pgmap v755: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:03:23.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:23 smithi155 ceph-mon[33846]: osdmap e492: 8 total, 8 up, 8 in 2024-03-21T13:03:23.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:23 smithi043 ceph-mon[30062]: pgmap v755: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:03:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:23 smithi043 ceph-mon[30062]: osdmap e492: 8 total, 8 up, 8 in 2024-03-21T13:03:23.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:23 smithi043 ceph-mon[41699]: pgmap v755: 356 pgs: 356 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:03:23.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:23 smithi043 ceph-mon[41699]: osdmap e492: 8 total, 8 up, 8 in 2024-03-21T13:03:24.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:24 smithi155 ceph-mon[33846]: osdmap e493: 8 total, 8 up, 8 in 2024-03-21T13:03:24.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:24 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:03:24.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:24 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:03:24.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:24 smithi043 ceph-mon[30062]: osdmap e493: 8 total, 8 up, 8 in 2024-03-21T13:03:24.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:24 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:03:24.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:24 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:03:24.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:24 smithi043 ceph-mon[41699]: osdmap e493: 8 total, 8 up, 8 in 2024-03-21T13:03:24.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:24 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:03:24.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:24 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:03:25.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:25 smithi155 ceph-mon[33846]: pgmap v758: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:03:25.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:25 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:03:25.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:25 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:03:25.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:25 smithi155 ceph-mon[33846]: osdmap e494: 8 total, 8 up, 8 in 2024-03-21T13:03:25.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:25 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:03:25.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:25 smithi043 ceph-mon[30062]: pgmap v758: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:03:25.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:25 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:03:25.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:25 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:03:25.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:25 smithi043 ceph-mon[30062]: osdmap e494: 8 total, 8 up, 8 in 2024-03-21T13:03:25.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:25 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:03:25.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:25 smithi043 ceph-mon[41699]: pgmap v758: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:03:25.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:25 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:03:25.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:25 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:03:25.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:25 smithi043 ceph-mon[41699]: osdmap e494: 8 total, 8 up, 8 in 2024-03-21T13:03:25.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:25 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:03:26.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:26 smithi155 ceph-mon[33846]: osdmap e495: 8 total, 8 up, 8 in 2024-03-21T13:03:26.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:26 smithi043 ceph-mon[30062]: osdmap e495: 8 total, 8 up, 8 in 2024-03-21T13:03:26.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:26 smithi043 ceph-mon[41699]: osdmap e495: 8 total, 8 up, 8 in 2024-03-21T13:03:27.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:27 smithi155 ceph-mon[33846]: pgmap v761: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:03:27.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:27 smithi043 ceph-mon[30062]: pgmap v761: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:03:27.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:27 smithi043 ceph-mon[41699]: pgmap v761: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:03:28.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:03:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:03:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:03:28.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:03:28 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:03:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:29 smithi155 ceph-mon[33846]: pgmap v762: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 940 B/s rd, 188 B/s wr, 1 op/s 2024-03-21T13:03:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:29 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:03:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:29 smithi043 ceph-mon[30062]: pgmap v762: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 940 B/s rd, 188 B/s wr, 1 op/s 2024-03-21T13:03:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:29 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:03:29.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:29 smithi043 ceph-mon[41699]: pgmap v762: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 940 B/s rd, 188 B/s wr, 1 op/s 2024-03-21T13:03:29.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:29 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:03:31.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:31 smithi155 ceph-mon[33846]: pgmap v763: 356 pgs: 356 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 795 B/s rd, 795 B/s wr, 2 op/s 2024-03-21T13:03:31.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:31 smithi043 ceph-mon[30062]: pgmap v763: 356 pgs: 356 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 795 B/s rd, 795 B/s wr, 2 op/s 2024-03-21T13:03:31.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:31 smithi043 ceph-mon[41699]: pgmap v763: 356 pgs: 356 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 795 B/s rd, 795 B/s wr, 2 op/s 2024-03-21T13:03:32.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:32 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:03:32.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:32 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:03:32.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:32 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:03:33.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:33 smithi155 ceph-mon[33846]: pgmap v764: 356 pgs: 356 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-03-21T13:03:33.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:33 smithi043 ceph-mon[30062]: pgmap v764: 356 pgs: 356 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-03-21T13:03:33.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:33 smithi043 ceph-mon[41699]: pgmap v764: 356 pgs: 356 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-03-21T13:03:34.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:34 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:03:34.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:34 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:03:34.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:34 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:03:35.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:35 smithi155 ceph-mon[33846]: pgmap v765: 356 pgs: 356 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 542 B/s wr, 2 op/s 2024-03-21T13:03:35.957 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:03:35 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:03:35.689+0000 7fe80cbf7700 -1 snap_mapper.add_oid found existing snaps mapped on 102:8bd5193b:test-rados-api-smithi043-81880-97::foo:21, removing 2024-03-21T13:03:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:35 smithi043 ceph-mon[30062]: pgmap v765: 356 pgs: 356 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 542 B/s wr, 2 op/s 2024-03-21T13:03:35.984 INFO:journalctl@ceph.osd.1.smithi043.stdout:Mar 21 13:03:35 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-1[52242]: 2024-03-21T13:03:35.689+0000 7fd9d5dbf700 -1 snap_mapper.add_oid found existing snaps mapped on 102:8bd5193b:test-rados-api-smithi043-81880-97::foo:21, removing 2024-03-21T13:03:35.984 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 13:03:35 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-2[58226]: 2024-03-21T13:03:35.689+0000 7f6a277ab700 -1 snap_mapper.add_oid found existing snaps mapped on 102:8bd5193b:test-rados-api-smithi043-81880-97::foo:21, removing 2024-03-21T13:03:35.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:35 smithi043 ceph-mon[41699]: pgmap v765: 356 pgs: 356 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 542 B/s wr, 2 op/s 2024-03-21T13:03:36.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:03:36.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:36 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:03:36.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-96"}]: dispatch 2024-03-21T13:03:36.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:36 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-96"}]: dispatch 2024-03-21T13:03:36.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:03:36.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:36 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:03:36.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-96"}]: dispatch 2024-03-21T13:03:36.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:36 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-96"}]: dispatch 2024-03-21T13:03:36.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:03:36.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:36 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:03:36.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-96"}]: dispatch 2024-03-21T13:03:36.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:36 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-96"}]: dispatch 2024-03-21T13:03:37.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:37 smithi155 ceph-mon[33846]: pgmap v766: 356 pgs: 356 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 981 B/s rd, 490 B/s wr, 2 op/s 2024-03-21T13:03:37.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:37 smithi155 ceph-mon[33846]: osdmap e496: 8 total, 8 up, 8 in 2024-03-21T13:03:37.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:37 smithi043 ceph-mon[30062]: pgmap v766: 356 pgs: 356 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 981 B/s rd, 490 B/s wr, 2 op/s 2024-03-21T13:03:37.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:37 smithi043 ceph-mon[30062]: osdmap e496: 8 total, 8 up, 8 in 2024-03-21T13:03:37.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:37 smithi043 ceph-mon[41699]: pgmap v766: 356 pgs: 356 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 981 B/s rd, 490 B/s wr, 2 op/s 2024-03-21T13:03:37.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:37 smithi043 ceph-mon[41699]: osdmap e496: 8 total, 8 up, 8 in 2024-03-21T13:03:38.489 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:03:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:03:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:03:38.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:38 smithi155 ceph-mon[33846]: osdmap e497: 8 total, 8 up, 8 in 2024-03-21T13:03:38.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:38 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:03:38.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:38 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:03:38.958 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:03:38 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:03:38.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:38 smithi043 ceph-mon[30062]: osdmap e497: 8 total, 8 up, 8 in 2024-03-21T13:03:38.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:38 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:03:38.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:38 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:03:38.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:38 smithi043 ceph-mon[41699]: osdmap e497: 8 total, 8 up, 8 in 2024-03-21T13:03:38.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:38 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:03:38.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:38 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:03:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:39 smithi043 ceph-mon[30062]: pgmap v769: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:03:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:39 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:03:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:39 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:03:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:39 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:03:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:39 smithi043 ceph-mon[30062]: osdmap e498: 8 total, 8 up, 8 in 2024-03-21T13:03:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:39 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:03:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:39 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:03:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:39 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-98", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:03:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:39 smithi043 ceph-mon[30062]: osdmap e499: 8 total, 8 up, 8 in 2024-03-21T13:03:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:39 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-98", "mode": "writeback"}]: dispatch 2024-03-21T13:03:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:39 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-98", "mode": "writeback"}]: dispatch 2024-03-21T13:03:39.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:39 smithi043 ceph-mon[41699]: pgmap v769: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:03:39.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:39 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:03:39.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:39 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:03:39.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:39 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:03:39.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:39 smithi043 ceph-mon[41699]: osdmap e498: 8 total, 8 up, 8 in 2024-03-21T13:03:39.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:39 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:03:39.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:39 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:03:39.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:39 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-98", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:03:39.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:39 smithi043 ceph-mon[41699]: osdmap e499: 8 total, 8 up, 8 in 2024-03-21T13:03:39.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:39 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-98", "mode": "writeback"}]: dispatch 2024-03-21T13:03:39.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:39 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-98", "mode": "writeback"}]: dispatch 2024-03-21T13:03:40.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:39 smithi155 ceph-mon[33846]: pgmap v769: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:03:40.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:39 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:03:40.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:39 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:03:40.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:39 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:03:40.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:39 smithi155 ceph-mon[33846]: osdmap e498: 8 total, 8 up, 8 in 2024-03-21T13:03:40.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:39 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:03:40.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:39 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:03:40.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:39 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-98", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:03:40.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:39 smithi155 ceph-mon[33846]: osdmap e499: 8 total, 8 up, 8 in 2024-03-21T13:03:40.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:39 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-98", "mode": "writeback"}]: dispatch 2024-03-21T13:03:40.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:39 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-98", "mode": "writeback"}]: dispatch 2024-03-21T13:03:40.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:40 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:03:40.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:40 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-98", "mode": "writeback"}]': finished 2024-03-21T13:03:40.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:40 smithi043 ceph-mon[30062]: osdmap e500: 8 total, 8 up, 8 in 2024-03-21T13:03:40.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:40 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-98"}]: dispatch 2024-03-21T13:03:40.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:40 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-98"}]: dispatch 2024-03-21T13:03:40.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:40 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:03:40.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:40 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-98", "mode": "writeback"}]': finished 2024-03-21T13:03:40.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:40 smithi043 ceph-mon[41699]: osdmap e500: 8 total, 8 up, 8 in 2024-03-21T13:03:40.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:40 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-98"}]: dispatch 2024-03-21T13:03:40.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:40 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-98"}]: dispatch 2024-03-21T13:03:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:40 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:03:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:40 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-98", "mode": "writeback"}]': finished 2024-03-21T13:03:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:40 smithi155 ceph-mon[33846]: osdmap e500: 8 total, 8 up, 8 in 2024-03-21T13:03:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:40 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-98"}]: dispatch 2024-03-21T13:03:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:40 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-98"}]: dispatch 2024-03-21T13:03:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:41 smithi155 ceph-mon[33846]: pgmap v772: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-03-21T13:03:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:41 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-98"}]': finished 2024-03-21T13:03:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:41 smithi155 ceph-mon[33846]: osdmap e501: 8 total, 8 up, 8 in 2024-03-21T13:03:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:03:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:41 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:03:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:41 smithi043 ceph-mon[30062]: pgmap v772: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-03-21T13:03:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:41 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-98"}]': finished 2024-03-21T13:03:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:41 smithi043 ceph-mon[30062]: osdmap e501: 8 total, 8 up, 8 in 2024-03-21T13:03:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:03:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:41 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:03:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:41 smithi043 ceph-mon[41699]: pgmap v772: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-03-21T13:03:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:41 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-98"}]': finished 2024-03-21T13:03:42.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:41 smithi043 ceph-mon[41699]: osdmap e501: 8 total, 8 up, 8 in 2024-03-21T13:03:42.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:03:42.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:41 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:03:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:42 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:03:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:42 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:03:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:42 smithi155 ceph-mon[33846]: osdmap e502: 8 total, 8 up, 8 in 2024-03-21T13:03:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-21T13:03:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:42 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-21T13:03:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:42 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:03:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:42 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:03:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:42 smithi043 ceph-mon[30062]: osdmap e502: 8 total, 8 up, 8 in 2024-03-21T13:03:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-21T13:03:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:42 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-21T13:03:43.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:42 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:03:43.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:42 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:03:43.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:42 smithi043 ceph-mon[41699]: osdmap e502: 8 total, 8 up, 8 in 2024-03-21T13:03:43.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-21T13:03:43.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:42 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-21T13:03:44.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:43 smithi155 ceph-mon[33846]: pgmap v775: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:03:44.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:43 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_count","val": "1"}]': finished 2024-03-21T13:03:44.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:43 smithi155 ceph-mon[33846]: osdmap e503: 8 total, 8 up, 8 in 2024-03-21T13:03:44.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:03:44.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:43 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:03:44.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:43 smithi043 ceph-mon[30062]: pgmap v775: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:03:44.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:43 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_count","val": "1"}]': finished 2024-03-21T13:03:44.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:43 smithi043 ceph-mon[30062]: osdmap e503: 8 total, 8 up, 8 in 2024-03-21T13:03:44.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:03:44.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:43 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:03:44.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:43 smithi043 ceph-mon[41699]: pgmap v775: 356 pgs: 32 creating+peering, 324 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:03:44.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:43 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_count","val": "1"}]': finished 2024-03-21T13:03:44.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:43 smithi043 ceph-mon[41699]: osdmap e503: 8 total, 8 up, 8 in 2024-03-21T13:03:44.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:03:44.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:43 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:03:45.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:45 smithi043 ceph-mon[30062]: pgmap v778: 356 pgs: 356 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:03:45.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:45 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:03:45.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:45 smithi043 ceph-mon[30062]: osdmap e504: 8 total, 8 up, 8 in 2024-03-21T13:03:45.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-21T13:03:45.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:45 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-21T13:03:45.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:45 smithi043 ceph-mon[41699]: pgmap v778: 356 pgs: 356 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:03:45.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:45 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:03:45.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:45 smithi043 ceph-mon[41699]: osdmap e504: 8 total, 8 up, 8 in 2024-03-21T13:03:45.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:45 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-21T13:03:45.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:45 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-21T13:03:46.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:45 smithi155 ceph-mon[33846]: pgmap v778: 356 pgs: 356 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:03:46.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:45 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:03:46.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:45 smithi155 ceph-mon[33846]: osdmap e504: 8 total, 8 up, 8 in 2024-03-21T13:03:46.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:45 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-21T13:03:46.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:45 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-21T13:03:46.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253f450 2024-03-21T13:03:46.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253f560 2024-03-21T13:03:46.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525400c0 2024-03-21T13:03:46.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552540590 2024-03-21T13:03:46.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525406c0 2024-03-21T13:03:46.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253fa30 2024-03-21T13:03:46.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253fb80 2024-03-21T13:03:46.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x55855253fcd0 2024-03-21T13:03:46.630 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552540810 2024-03-21T13:03:46.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552540ce0 2024-03-21T13:03:46.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552540e10 2024-03-21T13:03:46.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552541410 2024-03-21T13:03:46.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552540940 2024-03-21T13:03:46.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552540a70 2024-03-21T13:03:46.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552540ba0 2024-03-21T13:03:46.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552541520 2024-03-21T13:03:46.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552540f60 2024-03-21T13:03:46.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552541090 2024-03-21T13:03:46.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525411e0 2024-03-21T13:03:46.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552541670 2024-03-21T13:03:46.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552541b40 2024-03-21T13:03:46.631 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552541c70 2024-03-21T13:03:46.632 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525401f0 2024-03-21T13:03:46.632 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552540320 2024-03-21T13:03:46.632 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552540450 2024-03-21T13:03:46.632 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552541da0 2024-03-21T13:03:46.632 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552542270 2024-03-21T13:03:46.632 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525423a0 2024-03-21T13:03:46.632 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525417a0 2024-03-21T13:03:46.632 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525418d0 2024-03-21T13:03:46.632 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552541a00 2024-03-21T13:03:46.632 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525424f0 2024-03-21T13:03:46.632 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525429c0 2024-03-21T13:03:46.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552542b10 2024-03-21T13:03:46.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552541eb0 2024-03-21T13:03:46.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552542000 2024-03-21T13:03:46.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552542130 2024-03-21T13:03:46.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552542c40 2024-03-21T13:03:46.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552543110 2024-03-21T13:03:46.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552543240 2024-03-21T13:03:46.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552542600 2024-03-21T13:03:46.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552542750 2024-03-21T13:03:46.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552542880 2024-03-21T13:03:46.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552543390 2024-03-21T13:03:46.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552543860 2024-03-21T13:03:46.633 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552543990 2024-03-21T13:03:46.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552543f90 2024-03-21T13:03:46.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525434c0 2024-03-21T13:03:46.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525435f0 2024-03-21T13:03:46.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x558552543720 2024-03-21T13:03:46.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 0x5585525440c0 2024-03-21T13:03:46.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.634 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.635 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.635 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.635 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.635 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.635 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.635 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.635 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.635 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.635 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.635 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.635 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.636 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.637 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.637 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.637 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.637 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.637 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.637 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.637 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.637 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.637 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:03:46.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:46 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "target_max_objects","val": "250"}]': finished 2024-03-21T13:03:46.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:46 smithi155 ceph-mon[33846]: osdmap e505: 8 total, 8 up, 8 in 2024-03-21T13:03:46.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:03:46.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:46 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:03:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:46 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "target_max_objects","val": "250"}]': finished 2024-03-21T13:03:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:46 smithi043 ceph-mon[30062]: osdmap e505: 8 total, 8 up, 8 in 2024-03-21T13:03:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:03:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:46 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:03:47.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:46 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-98","var": "target_max_objects","val": "250"}]': finished 2024-03-21T13:03:47.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:46 smithi043 ceph-mon[41699]: osdmap e505: 8 total, 8 up, 8 in 2024-03-21T13:03:47.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:03:47.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:46 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:03:48.023 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:47 smithi043 ceph-mon[30062]: pgmap v781: 356 pgs: 356 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:03:48.023 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:47 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T13:03:48.023 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:47 smithi043 ceph-mon[30062]: osdmap e506: 8 total, 8 up, 8 in 2024-03-21T13:03:48.023 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-98"}]: dispatch 2024-03-21T13:03:48.023 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:47 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-98"}]: dispatch 2024-03-21T13:03:48.023 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:47 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:03:48.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:47 smithi043 ceph-mon[41699]: pgmap v781: 356 pgs: 356 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:03:48.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:47 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T13:03:48.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:47 smithi043 ceph-mon[41699]: osdmap e506: 8 total, 8 up, 8 in 2024-03-21T13:03:48.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-98"}]: dispatch 2024-03-21T13:03:48.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:47 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-98"}]: dispatch 2024-03-21T13:03:48.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:47 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:03:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:47 smithi155 ceph-mon[33846]: pgmap v781: 356 pgs: 356 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:03:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:47 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T13:03:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:47 smithi155 ceph-mon[33846]: osdmap e506: 8 total, 8 up, 8 in 2024-03-21T13:03:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-98"}]: dispatch 2024-03-21T13:03:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:47 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-98"}]: dispatch 2024-03-21T13:03:48.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:47 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:03:48.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:03:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:03:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:03:48.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:48 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-98"}]': finished 2024-03-21T13:03:48.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:48 smithi155 ceph-mon[33846]: osdmap e507: 8 total, 8 up, 8 in 2024-03-21T13:03:48.958 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:03:48 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:03:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:48 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-98"}]': finished 2024-03-21T13:03:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:48 smithi043 ceph-mon[30062]: osdmap e507: 8 total, 8 up, 8 in 2024-03-21T13:03:49.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:48 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-98"}]': finished 2024-03-21T13:03:49.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:48 smithi043 ceph-mon[41699]: osdmap e507: 8 total, 8 up, 8 in 2024-03-21T13:03:50.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:49 smithi155 ceph-mon[33846]: pgmap v784: 356 pgs: 356 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:03:50.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:49 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:03:50.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:49 smithi155 ceph-mon[33846]: osdmap e508: 8 total, 8 up, 8 in 2024-03-21T13:03:50.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:49 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:03:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:49 smithi043 ceph-mon[30062]: pgmap v784: 356 pgs: 356 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:03:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:49 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:03:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:49 smithi043 ceph-mon[30062]: osdmap e508: 8 total, 8 up, 8 in 2024-03-21T13:03:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:49 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:03:50.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:49 smithi043 ceph-mon[41699]: pgmap v784: 356 pgs: 356 active+clean; 8.4 MiB data, 611 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:03:50.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:49 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:03:50.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:49 smithi043 ceph-mon[41699]: osdmap e508: 8 total, 8 up, 8 in 2024-03-21T13:03:50.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:49 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:03:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:50 smithi155 ceph-mon[33846]: osdmap e509: 8 total, 8 up, 8 in 2024-03-21T13:03:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:50 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:03:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:50 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:03:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:50 smithi043 ceph-mon[30062]: osdmap e509: 8 total, 8 up, 8 in 2024-03-21T13:03:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:50 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:03:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:50 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:03:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:50 smithi043 ceph-mon[41699]: osdmap e509: 8 total, 8 up, 8 in 2024-03-21T13:03:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:50 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:03:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:50 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:03:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:51 smithi155 ceph-mon[33846]: pgmap v787: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:03:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:51 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:03:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:51 smithi155 ceph-mon[33846]: osdmap e510: 8 total, 8 up, 8 in 2024-03-21T13:03:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:51 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:03:52.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:51 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:03:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:51 smithi043 ceph-mon[30062]: pgmap v787: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:03:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:51 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:03:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:51 smithi043 ceph-mon[30062]: osdmap e510: 8 total, 8 up, 8 in 2024-03-21T13:03:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:51 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:03:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:51 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:03:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:51 smithi043 ceph-mon[41699]: pgmap v787: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:03:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:51 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:03:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:51 smithi043 ceph-mon[41699]: osdmap e510: 8 total, 8 up, 8 in 2024-03-21T13:03:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:51 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:03:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:51 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:03:53.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:52 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-100", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:03:53.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:52 smithi155 ceph-mon[33846]: osdmap e511: 8 total, 8 up, 8 in 2024-03-21T13:03:53.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-100"}]: dispatch 2024-03-21T13:03:53.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:52 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-100"}]: dispatch 2024-03-21T13:03:53.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:52 smithi155 ceph-mon[33846]: pgmap v790: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:03:53.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:52 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:03:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:52 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-100", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:03:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:52 smithi043 ceph-mon[30062]: osdmap e511: 8 total, 8 up, 8 in 2024-03-21T13:03:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-100"}]: dispatch 2024-03-21T13:03:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:52 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-100"}]: dispatch 2024-03-21T13:03:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:52 smithi043 ceph-mon[30062]: pgmap v790: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:03:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:52 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:03:53.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:52 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-100", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:03:53.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:52 smithi043 ceph-mon[41699]: osdmap e511: 8 total, 8 up, 8 in 2024-03-21T13:03:53.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-100"}]: dispatch 2024-03-21T13:03:53.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:52 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-100"}]: dispatch 2024-03-21T13:03:53.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:52 smithi043 ceph-mon[41699]: pgmap v790: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:03:53.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:52 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:03:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:53 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-100"}]': finished 2024-03-21T13:03:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:53 smithi155 ceph-mon[33846]: osdmap e512: 8 total, 8 up, 8 in 2024-03-21T13:03:54.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:53 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-100", "mode": "writeback"}]: dispatch 2024-03-21T13:03:54.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:53 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-100", "mode": "writeback"}]: dispatch 2024-03-21T13:03:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:53 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-100"}]': finished 2024-03-21T13:03:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:53 smithi043 ceph-mon[30062]: osdmap e512: 8 total, 8 up, 8 in 2024-03-21T13:03:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:53 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-100", "mode": "writeback"}]: dispatch 2024-03-21T13:03:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:53 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-100", "mode": "writeback"}]: dispatch 2024-03-21T13:03:54.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:53 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-6", "overlaypool": "test-rados-api-smithi043-81880-100"}]': finished 2024-03-21T13:03:54.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:53 smithi043 ceph-mon[41699]: osdmap e512: 8 total, 8 up, 8 in 2024-03-21T13:03:54.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:53 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-100", "mode": "writeback"}]: dispatch 2024-03-21T13:03:54.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:53 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-100", "mode": "writeback"}]: dispatch 2024-03-21T13:03:55.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:54 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:03:55.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:54 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-100", "mode": "writeback"}]': finished 2024-03-21T13:03:55.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:54 smithi155 ceph-mon[33846]: osdmap e513: 8 total, 8 up, 8 in 2024-03-21T13:03:55.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:54 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:03:55.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:54 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:03:55.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:54 smithi155 ceph-mon[33846]: pgmap v793: 356 pgs: 356 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:03:55.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:54 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:03:55.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:54 smithi155 ceph-mon[33846]: osdmap e514: 8 total, 8 up, 8 in 2024-03-21T13:03:55.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:54 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:03:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:54 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-100", "mode": "writeback"}]': finished 2024-03-21T13:03:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:54 smithi043 ceph-mon[30062]: osdmap e513: 8 total, 8 up, 8 in 2024-03-21T13:03:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:54 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:03:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:54 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:03:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:54 smithi043 ceph-mon[30062]: pgmap v793: 356 pgs: 356 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:03:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:54 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:03:55.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:54 smithi043 ceph-mon[30062]: osdmap e514: 8 total, 8 up, 8 in 2024-03-21T13:03:55.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:54 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:03:55.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:54 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-100", "mode": "writeback"}]': finished 2024-03-21T13:03:55.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:54 smithi043 ceph-mon[41699]: osdmap e513: 8 total, 8 up, 8 in 2024-03-21T13:03:55.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:54 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:03:55.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:54 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:03:55.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:54 smithi043 ceph-mon[41699]: pgmap v793: 356 pgs: 356 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:03:55.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:54 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:03:55.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:54 smithi043 ceph-mon[41699]: osdmap e514: 8 total, 8 up, 8 in 2024-03-21T13:03:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:55 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:03:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:55 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:03:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:55 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:03:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:55 smithi155 ceph-mon[33846]: osdmap e515: 8 total, 8 up, 8 in 2024-03-21T13:03:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:55 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:03:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:55 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:03:56.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:55 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:03:56.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:55 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:03:56.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:55 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:03:56.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:55 smithi043 ceph-mon[30062]: osdmap e515: 8 total, 8 up, 8 in 2024-03-21T13:03:56.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:55 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:03:56.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:55 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:03:56.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:55 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:03:56.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:55 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:03:56.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:55 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:03:56.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:55 smithi043 ceph-mon[41699]: osdmap e515: 8 total, 8 up, 8 in 2024-03-21T13:03:56.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:55 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:03:56.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:55 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:03:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:56 smithi155 ceph-mon[33846]: pgmap v796: 356 pgs: 356 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:03:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:56 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:03:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:56 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-21T13:03:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:56 smithi155 ceph-mon[33846]: osdmap e516: 8 total, 8 up, 8 in 2024-03-21T13:03:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:56 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-21T13:03:57.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:56 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-21T13:03:57.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:56 smithi043 ceph-mon[30062]: pgmap v796: 356 pgs: 356 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:03:57.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:56 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:03:57.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:56 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-21T13:03:57.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:56 smithi043 ceph-mon[30062]: osdmap e516: 8 total, 8 up, 8 in 2024-03-21T13:03:57.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:56 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-21T13:03:57.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:56 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-21T13:03:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:56 smithi043 ceph-mon[41699]: pgmap v796: 356 pgs: 356 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:03:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:56 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:03:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:56 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-21T13:03:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:56 smithi043 ceph-mon[41699]: osdmap e516: 8 total, 8 up, 8 in 2024-03-21T13:03:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:56 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-21T13:03:57.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:56 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-21T13:03:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:57 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:03:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:57 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-03-21T13:03:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:57 smithi155 ceph-mon[33846]: osdmap e517: 8 total, 8 up, 8 in 2024-03-21T13:03:58.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:57 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:03:58.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:57 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-03-21T13:03:58.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:57 smithi043 ceph-mon[30062]: osdmap e517: 8 total, 8 up, 8 in 2024-03-21T13:03:58.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:03:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:03:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:03:58.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:57 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:03:58.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:57 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-03-21T13:03:58.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:57 smithi043 ceph-mon[41699]: osdmap e517: 8 total, 8 up, 8 in 2024-03-21T13:03:58.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:03:58 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:03:58.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:58 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:03:58.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:58 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:03:58.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:58 smithi155 ceph-mon[33846]: pgmap v799: 356 pgs: 356 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:03:58.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:58 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:03:59.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:58 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:03:59.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:58 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:03:59.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:58 smithi043 ceph-mon[41699]: pgmap v799: 356 pgs: 356 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:03:59.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:58 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:03:59.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:58 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:03:59.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:58 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:03:59.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:58 smithi043 ceph-mon[30062]: pgmap v799: 356 pgs: 356 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:03:59.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:58 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:04:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:59 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T13:04:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:59 smithi155 ceph-mon[33846]: osdmap e518: 8 total, 8 up, 8 in 2024-03-21T13:04:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:59 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-100"}]: dispatch 2024-03-21T13:04:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:03:59 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-100"}]: dispatch 2024-03-21T13:04:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:59 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T13:04:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:59 smithi043 ceph-mon[30062]: osdmap e518: 8 total, 8 up, 8 in 2024-03-21T13:04:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:59 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-100"}]: dispatch 2024-03-21T13:04:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:03:59 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-100"}]: dispatch 2024-03-21T13:04:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:59 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]': finished 2024-03-21T13:04:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:59 smithi043 ceph-mon[41699]: osdmap e518: 8 total, 8 up, 8 in 2024-03-21T13:04:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:59 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-100"}]: dispatch 2024-03-21T13:04:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:03:59 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-100"}]: dispatch 2024-03-21T13:04:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:00 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-100"}]': finished 2024-03-21T13:04:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:00 smithi155 ceph-mon[33846]: osdmap e519: 8 total, 8 up, 8 in 2024-03-21T13:04:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:00 smithi155 ceph-mon[33846]: pgmap v802: 356 pgs: 356 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:04:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:00 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:04:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:00 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-100"}]': finished 2024-03-21T13:04:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:00 smithi043 ceph-mon[30062]: osdmap e519: 8 total, 8 up, 8 in 2024-03-21T13:04:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:00 smithi043 ceph-mon[30062]: pgmap v802: 356 pgs: 356 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:04:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:00 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:04:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:00 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-100"}]': finished 2024-03-21T13:04:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:00 smithi043 ceph-mon[41699]: osdmap e519: 8 total, 8 up, 8 in 2024-03-21T13:04:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:00 smithi043 ceph-mon[41699]: pgmap v802: 356 pgs: 356 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:04:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:00 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:04:02.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:01 smithi155 ceph-mon[33846]: osdmap e520: 8 total, 8 up, 8 in 2024-03-21T13:04:02.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:01 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:04:02.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:01 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:04:02.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:01 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:04:02.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:01 smithi043 ceph-mon[30062]: osdmap e520: 8 total, 8 up, 8 in 2024-03-21T13:04:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:01 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:04:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:01 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:04:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:01 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:04:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:01 smithi043 ceph-mon[41699]: osdmap e520: 8 total, 8 up, 8 in 2024-03-21T13:04:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:01 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:04:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:01 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:04:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:01 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:04:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:02 smithi155 ceph-mon[33846]: osdmap e521: 8 total, 8 up, 8 in 2024-03-21T13:04:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:02 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:02 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:02 smithi155 ceph-mon[33846]: pgmap v805: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:04:03.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:02 smithi043 ceph-mon[30062]: osdmap e521: 8 total, 8 up, 8 in 2024-03-21T13:04:03.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:02 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:03.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:02 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:03.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:02 smithi043 ceph-mon[30062]: pgmap v805: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:04:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:02 smithi043 ceph-mon[41699]: osdmap e521: 8 total, 8 up, 8 in 2024-03-21T13:04:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:02 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:02 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:02 smithi043 ceph-mon[41699]: pgmap v805: 356 pgs: 32 unknown, 324 active+clean; 8.4 MiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:04:04.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:03 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:04:04.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:03 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:04:04.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:03 smithi155 ceph-mon[33846]: osdmap e522: 8 total, 8 up, 8 in 2024-03-21T13:04:04.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:03 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:04:04.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:03 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:04:04.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:03 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:04:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:03 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:04:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:03 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:04:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:03 smithi043 ceph-mon[30062]: osdmap e522: 8 total, 8 up, 8 in 2024-03-21T13:04:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:03 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:04:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:03 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:04:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:03 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:04:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:03 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:04:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:03 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:04:04.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:03 smithi043 ceph-mon[41699]: osdmap e522: 8 total, 8 up, 8 in 2024-03-21T13:04:04.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:03 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-21T13:04:04.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:03 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:04:04.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:03 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-21T13:04:05.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:04 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:04:05.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:04 smithi155 ceph-mon[33846]: osdmap e523: 8 total, 8 up, 8 in 2024-03-21T13:04:05.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:04:05.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:04 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:04:05.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:04 smithi155 ceph-mon[33846]: pgmap v808: 356 pgs: 356 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:05.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:04 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:04:05.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:04 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:04:05.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:04 smithi155 ceph-mon[33846]: osdmap e524: 8 total, 8 up, 8 in 2024-03-21T13:04:05.110 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:04:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:04 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:04:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:04 smithi043 ceph-mon[30062]: osdmap e523: 8 total, 8 up, 8 in 2024-03-21T13:04:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:04:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:04 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:04:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:04 smithi043 ceph-mon[30062]: pgmap v808: 356 pgs: 356 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:04 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:04:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:04 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:04:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:04 smithi043 ceph-mon[30062]: osdmap e524: 8 total, 8 up, 8 in 2024-03-21T13:04:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:04:05.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:04 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-21T13:04:05.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:04 smithi043 ceph-mon[41699]: osdmap e523: 8 total, 8 up, 8 in 2024-03-21T13:04:05.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:04:05.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:04 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-21T13:04:05.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:04 smithi043 ceph-mon[41699]: pgmap v808: 356 pgs: 356 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:05.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:04 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:04:05.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:04 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-21T13:04:05.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:04 smithi043 ceph-mon[41699]: osdmap e524: 8 total, 8 up, 8 in 2024-03-21T13:04:05.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:04:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:05 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:04:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:05 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:04:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:05 smithi155 ceph-mon[33846]: osdmap e525: 8 total, 8 up, 8 in 2024-03-21T13:04:06.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:05 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:04:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:05 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:04:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:05 smithi043 ceph-mon[30062]: osdmap e525: 8 total, 8 up, 8 in 2024-03-21T13:04:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:05 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-21T13:04:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:05 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-21T13:04:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:05 smithi043 ceph-mon[41699]: osdmap e525: 8 total, 8 up, 8 in 2024-03-21T13:04:07.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:06 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:04:07.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:06 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:04:07.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:06 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-102"}]: dispatch 2024-03-21T13:04:07.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:06 smithi155 ceph-mon[33846]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-102"}]: dispatch 2024-03-21T13:04:07.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:06 smithi155 ceph-mon[33846]: pgmap v811: 356 pgs: 356 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:07.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:06 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:04:07.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:06 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:04:07.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:06 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-102"}]: dispatch 2024-03-21T13:04:07.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:06 smithi043 ceph-mon[30062]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-102"}]: dispatch 2024-03-21T13:04:07.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:06 smithi043 ceph-mon[30062]: pgmap v811: 356 pgs: 356 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:07.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:06 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:04:07.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:06 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-6"}]: dispatch 2024-03-21T13:04:07.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:06 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4290855917' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-102"}]: dispatch 2024-03-21T13:04:07.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:06 smithi043 ceph-mon[41699]: from='client.35495 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-6", "tierpool": "test-rados-api-smithi043-81880-102"}]: dispatch 2024-03-21T13:04:07.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:06 smithi043 ceph-mon[41699]: pgmap v811: 356 pgs: 356 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:08.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:07 smithi043 ceph-mon[30062]: osdmap e526: 8 total, 8 up, 8 in 2024-03-21T13:04:08.135 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:04:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:04:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:04:08.135 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:07 smithi043 ceph-mon[41699]: osdmap e526: 8 total, 8 up, 8 in 2024-03-21T13:04:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:07 smithi155 ceph-mon[33846]: osdmap e526: 8 total, 8 up, 8 in 2024-03-21T13:04:08.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:08 smithi155 ceph-mon[33846]: osdmap e527: 8 total, 8 up, 8 in 2024-03-21T13:04:08.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:08 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3927405983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:08.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:08 smithi155 ceph-mon[33846]: from='client.36338 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:08.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:08 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3927405983' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:08.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:08 smithi155 ceph-mon[33846]: from='client.36338 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:08.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:08 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3927405983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-81880-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:04:08.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:08 smithi155 ceph-mon[33846]: from='client.36338 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-81880-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:04:08.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:08 smithi155 ceph-mon[33846]: pgmap v814: 292 pgs: 292 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:04:08.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:08 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:04:08.961 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:04:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:04:09.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:08 smithi043 ceph-mon[41699]: osdmap e527: 8 total, 8 up, 8 in 2024-03-21T13:04:09.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:08 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3927405983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:09.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:08 smithi043 ceph-mon[41699]: from='client.36338 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:09.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:08 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3927405983' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:09.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:08 smithi043 ceph-mon[41699]: from='client.36338 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:09.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:08 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3927405983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-81880-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:04:09.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:08 smithi043 ceph-mon[41699]: from='client.36338 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-81880-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:04:09.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:08 smithi043 ceph-mon[41699]: pgmap v814: 292 pgs: 292 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:04:09.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:08 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:04:09.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:08 smithi043 ceph-mon[30062]: osdmap e527: 8 total, 8 up, 8 in 2024-03-21T13:04:09.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:08 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3927405983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:09.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:08 smithi043 ceph-mon[30062]: from='client.36338 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:09.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:08 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3927405983' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:09.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:08 smithi043 ceph-mon[30062]: from='client.36338 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:09.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:08 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3927405983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-81880-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:04:09.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:08 smithi043 ceph-mon[30062]: from='client.36338 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-81880-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:04:09.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:08 smithi043 ceph-mon[30062]: pgmap v814: 292 pgs: 292 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:04:09.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:08 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:04:10.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:10 smithi155 ceph-mon[33846]: from='client.36338 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-81880-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:04:10.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:10 smithi155 ceph-mon[33846]: osdmap e528: 8 total, 8 up, 8 in 2024-03-21T13:04:10.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3927405983' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-81880-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:10.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:10 smithi155 ceph-mon[33846]: from='client.36338 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-81880-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:10.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:10 smithi043 ceph-mon[30062]: from='client.36338 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-81880-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:04:10.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:10 smithi043 ceph-mon[30062]: osdmap e528: 8 total, 8 up, 8 in 2024-03-21T13:04:10.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3927405983' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-81880-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:10.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:10 smithi043 ceph-mon[30062]: from='client.36338 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-81880-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:10.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:10 smithi043 ceph-mon[41699]: from='client.36338 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi043-81880-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:04:10.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:10 smithi043 ceph-mon[41699]: osdmap e528: 8 total, 8 up, 8 in 2024-03-21T13:04:10.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3927405983' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-81880-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:10.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:10 smithi043 ceph-mon[41699]: from='client.36338 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-81880-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:11.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:11 smithi155 ceph-mon[33846]: pgmap v817: 292 pgs: 292 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:04:11.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:11 smithi155 ceph-mon[33846]: osdmap e529: 8 total, 8 up, 8 in 2024-03-21T13:04:11.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:11 smithi155 ceph-mon[33846]: from='client.36338 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-81880-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-81880-104"}]': finished 2024-03-21T13:04:11.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:11 smithi155 ceph-mon[33846]: osdmap e530: 8 total, 8 up, 8 in 2024-03-21T13:04:11.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:11 smithi043 ceph-mon[30062]: pgmap v817: 292 pgs: 292 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:04:11.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:11 smithi043 ceph-mon[30062]: osdmap e529: 8 total, 8 up, 8 in 2024-03-21T13:04:11.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:11 smithi043 ceph-mon[30062]: from='client.36338 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-81880-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-81880-104"}]': finished 2024-03-21T13:04:11.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:11 smithi043 ceph-mon[30062]: osdmap e530: 8 total, 8 up, 8 in 2024-03-21T13:04:11.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:11 smithi043 ceph-mon[41699]: pgmap v817: 292 pgs: 292 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:04:11.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:11 smithi043 ceph-mon[41699]: osdmap e529: 8 total, 8 up, 8 in 2024-03-21T13:04:11.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:11 smithi043 ceph-mon[41699]: from='client.36338 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi043-81880-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi043-81880-104"}]': finished 2024-03-21T13:04:11.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:11 smithi043 ceph-mon[41699]: osdmap e530: 8 total, 8 up, 8 in 2024-03-21T13:04:13.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:12 smithi155 ceph-mon[33846]: osdmap e531: 8 total, 8 up, 8 in 2024-03-21T13:04:13.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:12 smithi155 ceph-mon[33846]: pgmap v820: 300 pgs: 8 unknown, 292 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:13.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:12 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:04:13.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:12 smithi043 ceph-mon[30062]: osdmap e531: 8 total, 8 up, 8 in 2024-03-21T13:04:13.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:12 smithi043 ceph-mon[30062]: pgmap v820: 300 pgs: 8 unknown, 292 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:13.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:12 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:04:13.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:12 smithi043 ceph-mon[41699]: osdmap e531: 8 total, 8 up, 8 in 2024-03-21T13:04:13.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:12 smithi043 ceph-mon[41699]: pgmap v820: 300 pgs: 8 unknown, 292 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:13.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:12 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:04:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:13 smithi155 ceph-mon[33846]: osdmap e532: 8 total, 8 up, 8 in 2024-03-21T13:04:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:13 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:14.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:13 smithi043 ceph-mon[30062]: osdmap e532: 8 total, 8 up, 8 in 2024-03-21T13:04:14.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:13 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:14.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:13 smithi043 ceph-mon[41699]: osdmap e532: 8 total, 8 up, 8 in 2024-03-21T13:04:14.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:13 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:15.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:14 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:04:15.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:14 smithi155 ceph-mon[33846]: osdmap e533: 8 total, 8 up, 8 in 2024-03-21T13:04:15.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:14 smithi155 ceph-mon[33846]: pgmap v823: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:04:15.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:14 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:04:15.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:14 smithi043 ceph-mon[30062]: osdmap e533: 8 total, 8 up, 8 in 2024-03-21T13:04:15.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:14 smithi043 ceph-mon[30062]: pgmap v823: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:04:15.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:14 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:04:15.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:14 smithi043 ceph-mon[41699]: osdmap e533: 8 total, 8 up, 8 in 2024-03-21T13:04:15.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:14 smithi043 ceph-mon[41699]: pgmap v823: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:04:16.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:15 smithi155 ceph-mon[33846]: osdmap e534: 8 total, 8 up, 8 in 2024-03-21T13:04:16.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:15 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:16.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:15 smithi043 ceph-mon[30062]: osdmap e534: 8 total, 8 up, 8 in 2024-03-21T13:04:16.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:15 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:16.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:15 smithi043 ceph-mon[41699]: osdmap e534: 8 total, 8 up, 8 in 2024-03-21T13:04:16.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:15 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:17.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:16 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:04:17.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:16 smithi155 ceph-mon[33846]: osdmap e535: 8 total, 8 up, 8 in 2024-03-21T13:04:17.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:16 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-107", "tierpool": "test-rados-api-smithi043-81880-107-cache"}]: dispatch 2024-03-21T13:04:17.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:16 smithi155 ceph-mon[33846]: pgmap v826: 364 pgs: 32 unknown, 32 creating+peering, 300 active+clean; 582 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:04:17.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:16 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-107", "tierpool": "test-rados-api-smithi043-81880-107-cache"}]': finished 2024-03-21T13:04:17.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:16 smithi155 ceph-mon[33846]: osdmap e536: 8 total, 8 up, 8 in 2024-03-21T13:04:17.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:16 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-107", "overlaypool": "test-rados-api-smithi043-81880-107-cache"}]: dispatch 2024-03-21T13:04:17.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:16 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:04:17.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:16 smithi043 ceph-mon[30062]: osdmap e535: 8 total, 8 up, 8 in 2024-03-21T13:04:17.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:16 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-107", "tierpool": "test-rados-api-smithi043-81880-107-cache"}]: dispatch 2024-03-21T13:04:17.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:16 smithi043 ceph-mon[30062]: pgmap v826: 364 pgs: 32 unknown, 32 creating+peering, 300 active+clean; 582 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:04:17.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:16 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-107", "tierpool": "test-rados-api-smithi043-81880-107-cache"}]': finished 2024-03-21T13:04:17.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:16 smithi043 ceph-mon[30062]: osdmap e536: 8 total, 8 up, 8 in 2024-03-21T13:04:17.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:16 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-107", "overlaypool": "test-rados-api-smithi043-81880-107-cache"}]: dispatch 2024-03-21T13:04:17.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:16 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:04:17.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:16 smithi043 ceph-mon[41699]: osdmap e535: 8 total, 8 up, 8 in 2024-03-21T13:04:17.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:16 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-107", "tierpool": "test-rados-api-smithi043-81880-107-cache"}]: dispatch 2024-03-21T13:04:17.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:16 smithi043 ceph-mon[41699]: pgmap v826: 364 pgs: 32 unknown, 32 creating+peering, 300 active+clean; 582 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:04:17.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:16 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-107", "tierpool": "test-rados-api-smithi043-81880-107-cache"}]': finished 2024-03-21T13:04:17.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:16 smithi043 ceph-mon[41699]: osdmap e536: 8 total, 8 up, 8 in 2024-03-21T13:04:17.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:16 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-107", "overlaypool": "test-rados-api-smithi043-81880-107-cache"}]: dispatch 2024-03-21T13:04:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:17 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:04:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:17 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-107", "overlaypool": "test-rados-api-smithi043-81880-107-cache"}]': finished 2024-03-21T13:04:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:17 smithi155 ceph-mon[33846]: osdmap e537: 8 total, 8 up, 8 in 2024-03-21T13:04:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:17 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-107-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:04:18.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:04:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:04:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:04:18.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:17 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:04:18.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:17 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-107", "overlaypool": "test-rados-api-smithi043-81880-107-cache"}]': finished 2024-03-21T13:04:18.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:17 smithi043 ceph-mon[30062]: osdmap e537: 8 total, 8 up, 8 in 2024-03-21T13:04:18.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:17 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-107-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:04:18.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:17 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:04:18.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:17 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-107", "overlaypool": "test-rados-api-smithi043-81880-107-cache"}]': finished 2024-03-21T13:04:18.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:17 smithi043 ceph-mon[41699]: osdmap e537: 8 total, 8 up, 8 in 2024-03-21T13:04:18.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:17 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-107-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:04:18.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:04:18 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:04:18.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:18 smithi155 ceph-mon[33846]: pgmap v829: 364 pgs: 32 unknown, 32 creating+peering, 300 active+clean; 582 KiB data, 598 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:04:18.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:18 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:04:18.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:18 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:04:18.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:18 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-107-cache", "mode": "writeback"}]': finished 2024-03-21T13:04:18.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:18 smithi155 ceph-mon[33846]: osdmap e538: 8 total, 8 up, 8 in 2024-03-21T13:04:19.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:18 smithi043 ceph-mon[30062]: pgmap v829: 364 pgs: 32 unknown, 32 creating+peering, 300 active+clean; 582 KiB data, 598 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:04:19.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:18 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:04:19.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:18 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:04:19.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:18 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-107-cache", "mode": "writeback"}]': finished 2024-03-21T13:04:19.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:18 smithi043 ceph-mon[30062]: osdmap e538: 8 total, 8 up, 8 in 2024-03-21T13:04:19.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:18 smithi043 ceph-mon[41699]: pgmap v829: 364 pgs: 32 unknown, 32 creating+peering, 300 active+clean; 582 KiB data, 598 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:04:19.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:18 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:04:19.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:18 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:04:19.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:18 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-107-cache", "mode": "writeback"}]': finished 2024-03-21T13:04:19.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:18 smithi043 ceph-mon[41699]: osdmap e538: 8 total, 8 up, 8 in 2024-03-21T13:04:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:19 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-107"}]: dispatch 2024-03-21T13:04:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:19 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:04:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:19 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:04:20.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:19 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-107"}]: dispatch 2024-03-21T13:04:20.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:19 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:04:20.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:19 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:04:20.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:19 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-107"}]: dispatch 2024-03-21T13:04:20.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:19 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:04:20.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:19 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:04:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:20 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-107"}]': finished 2024-03-21T13:04:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:20 smithi155 ceph-mon[33846]: osdmap e539: 8 total, 8 up, 8 in 2024-03-21T13:04:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:20 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-107", "tierpool": "test-rados-api-smithi043-81880-107-cache"}]: dispatch 2024-03-21T13:04:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:20 smithi155 ceph-mon[33846]: pgmap v832: 364 pgs: 364 active+clean; 582 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:20 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-107"}]': finished 2024-03-21T13:04:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:20 smithi043 ceph-mon[30062]: osdmap e539: 8 total, 8 up, 8 in 2024-03-21T13:04:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:20 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-107", "tierpool": "test-rados-api-smithi043-81880-107-cache"}]: dispatch 2024-03-21T13:04:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:20 smithi043 ceph-mon[30062]: pgmap v832: 364 pgs: 364 active+clean; 582 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:20 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-107"}]': finished 2024-03-21T13:04:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:20 smithi043 ceph-mon[41699]: osdmap e539: 8 total, 8 up, 8 in 2024-03-21T13:04:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:20 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-107", "tierpool": "test-rados-api-smithi043-81880-107-cache"}]: dispatch 2024-03-21T13:04:21.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:20 smithi043 ceph-mon[41699]: pgmap v832: 364 pgs: 364 active+clean; 582 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:22.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:21 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:04:22.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:21 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-107", "tierpool": "test-rados-api-smithi043-81880-107-cache"}]': finished 2024-03-21T13:04:22.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:21 smithi155 ceph-mon[33846]: osdmap e540: 8 total, 8 up, 8 in 2024-03-21T13:04:22.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:21 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:04:22.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:21 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-107", "tierpool": "test-rados-api-smithi043-81880-107-cache"}]': finished 2024-03-21T13:04:22.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:21 smithi043 ceph-mon[30062]: osdmap e540: 8 total, 8 up, 8 in 2024-03-21T13:04:22.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:21 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:04:22.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:21 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3521105934' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-107", "tierpool": "test-rados-api-smithi043-81880-107-cache"}]': finished 2024-03-21T13:04:22.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:21 smithi043 ceph-mon[41699]: osdmap e540: 8 total, 8 up, 8 in 2024-03-21T13:04:23.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:22 smithi155 ceph-mon[33846]: osdmap e541: 8 total, 8 up, 8 in 2024-03-21T13:04:23.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:22 smithi155 ceph-mon[33846]: pgmap v835: 332 pgs: 332 active+clean; 582 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:23.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:22 smithi043 ceph-mon[30062]: osdmap e541: 8 total, 8 up, 8 in 2024-03-21T13:04:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:22 smithi043 ceph-mon[30062]: pgmap v835: 332 pgs: 332 active+clean; 582 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:23.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:22 smithi043 ceph-mon[41699]: osdmap e541: 8 total, 8 up, 8 in 2024-03-21T13:04:23.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:22 smithi043 ceph-mon[41699]: pgmap v835: 332 pgs: 332 active+clean; 582 KiB data, 598 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:23 smithi155 ceph-mon[33846]: osdmap e542: 8 total, 8 up, 8 in 2024-03-21T13:04:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:23 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:23 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:23 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:23 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:24.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:23 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81880-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:04:24.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:23 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81880-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:04:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:23 smithi043 ceph-mon[30062]: osdmap e542: 8 total, 8 up, 8 in 2024-03-21T13:04:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:23 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:23 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:23 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:23 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:23 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81880-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:04:24.237 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:23 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81880-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:04:24.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:23 smithi043 ceph-mon[41699]: osdmap e542: 8 total, 8 up, 8 in 2024-03-21T13:04:24.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:23 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:24.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:23 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:24.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:23 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:24.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:23 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:24.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:23 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81880-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:04:24.238 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:23 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81880-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:04:25.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:25 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81880-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:04:25.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:25 smithi043 ceph-mon[30062]: osdmap e543: 8 total, 8 up, 8 in 2024-03-21T13:04:25.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:25 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81880-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:25.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:25 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81880-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:25.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:25 smithi043 ceph-mon[30062]: pgmap v838: 300 pgs: 300 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:25.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:25 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81880-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:04:25.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:25 smithi043 ceph-mon[41699]: osdmap e543: 8 total, 8 up, 8 in 2024-03-21T13:04:25.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:25 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81880-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:25.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:25 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81880-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:25.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:25 smithi043 ceph-mon[41699]: pgmap v838: 300 pgs: 300 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:25.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:25 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81880-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:04:25.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:25 smithi155 ceph-mon[33846]: osdmap e543: 8 total, 8 up, 8 in 2024-03-21T13:04:25.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:25 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81880-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:25.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:25 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81880-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:25.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:25 smithi155 ceph-mon[33846]: pgmap v838: 300 pgs: 300 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:26.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:26 smithi043 ceph-mon[30062]: osdmap e544: 8 total, 8 up, 8 in 2024-03-21T13:04:26.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:26 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81880-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81880-109"}]': finished 2024-03-21T13:04:26.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:26 smithi043 ceph-mon[30062]: osdmap e545: 8 total, 8 up, 8 in 2024-03-21T13:04:26.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:26 smithi043 ceph-mon[41699]: osdmap e544: 8 total, 8 up, 8 in 2024-03-21T13:04:26.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:26 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81880-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81880-109"}]': finished 2024-03-21T13:04:26.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:26 smithi043 ceph-mon[41699]: osdmap e545: 8 total, 8 up, 8 in 2024-03-21T13:04:26.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:26 smithi155 ceph-mon[33846]: osdmap e544: 8 total, 8 up, 8 in 2024-03-21T13:04:26.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:26 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81880-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81880-109"}]': finished 2024-03-21T13:04:26.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:26 smithi155 ceph-mon[33846]: osdmap e545: 8 total, 8 up, 8 in 2024-03-21T13:04:27.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:27 smithi043 ceph-mon[30062]: pgmap v841: 308 pgs: 8 unknown, 300 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:27.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:27 smithi043 ceph-mon[30062]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:04:27.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:27 smithi043 ceph-mon[30062]: osdmap e546: 8 total, 8 up, 8 in 2024-03-21T13:04:27.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:27 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:27.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:27 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:27.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:27 smithi043 ceph-mon[41699]: pgmap v841: 308 pgs: 8 unknown, 300 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:27.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:27 smithi043 ceph-mon[41699]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:04:27.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:27 smithi043 ceph-mon[41699]: osdmap e546: 8 total, 8 up, 8 in 2024-03-21T13:04:27.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:27 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:27.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:27 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:27.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:27 smithi155 ceph-mon[33846]: pgmap v841: 308 pgs: 8 unknown, 300 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:27.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:27 smithi155 ceph-mon[33846]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:04:27.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:27 smithi155 ceph-mon[33846]: osdmap e546: 8 total, 8 up, 8 in 2024-03-21T13:04:27.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:27 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:27.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:27 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:28.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:04:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:04:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:04:28.964 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:04:28 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:04:29.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:28 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:04:29.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:28 smithi043 ceph-mon[30062]: osdmap e547: 8 total, 8 up, 8 in 2024-03-21T13:04:29.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-109", "tierpool": "test-rados-api-smithi043-81880-109-cache"}]: dispatch 2024-03-21T13:04:29.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:28 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-109", "tierpool": "test-rados-api-smithi043-81880-109-cache"}]: dispatch 2024-03-21T13:04:29.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:28 smithi043 ceph-mon[30062]: pgmap v844: 340 pgs: 40 unknown, 300 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:04:29.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:28 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:04:29.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:28 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:04:29.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:28 smithi043 ceph-mon[41699]: osdmap e547: 8 total, 8 up, 8 in 2024-03-21T13:04:29.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-109", "tierpool": "test-rados-api-smithi043-81880-109-cache"}]: dispatch 2024-03-21T13:04:29.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:28 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-109", "tierpool": "test-rados-api-smithi043-81880-109-cache"}]: dispatch 2024-03-21T13:04:29.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:28 smithi043 ceph-mon[41699]: pgmap v844: 340 pgs: 40 unknown, 300 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:04:29.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:28 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:04:29.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:28 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:04:29.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:28 smithi155 ceph-mon[33846]: osdmap e547: 8 total, 8 up, 8 in 2024-03-21T13:04:29.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-109", "tierpool": "test-rados-api-smithi043-81880-109-cache"}]: dispatch 2024-03-21T13:04:29.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:28 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-109", "tierpool": "test-rados-api-smithi043-81880-109-cache"}]: dispatch 2024-03-21T13:04:29.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:28 smithi155 ceph-mon[33846]: pgmap v844: 340 pgs: 40 unknown, 300 active+clean; 582 KiB data, 599 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:04:29.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:28 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:04:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:29 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-109", "tierpool": "test-rados-api-smithi043-81880-109-cache"}]': finished 2024-03-21T13:04:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:29 smithi043 ceph-mon[30062]: osdmap e548: 8 total, 8 up, 8 in 2024-03-21T13:04:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:29 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-109", "overlaypool": "test-rados-api-smithi043-81880-109-cache"}]: dispatch 2024-03-21T13:04:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:29 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-109", "overlaypool": "test-rados-api-smithi043-81880-109-cache"}]: dispatch 2024-03-21T13:04:30.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:29 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-109", "tierpool": "test-rados-api-smithi043-81880-109-cache"}]': finished 2024-03-21T13:04:30.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:29 smithi043 ceph-mon[41699]: osdmap e548: 8 total, 8 up, 8 in 2024-03-21T13:04:30.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:29 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-109", "overlaypool": "test-rados-api-smithi043-81880-109-cache"}]: dispatch 2024-03-21T13:04:30.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:29 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-109", "overlaypool": "test-rados-api-smithi043-81880-109-cache"}]: dispatch 2024-03-21T13:04:30.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:29 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-109", "tierpool": "test-rados-api-smithi043-81880-109-cache"}]': finished 2024-03-21T13:04:30.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:29 smithi155 ceph-mon[33846]: osdmap e548: 8 total, 8 up, 8 in 2024-03-21T13:04:30.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:29 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-109", "overlaypool": "test-rados-api-smithi043-81880-109-cache"}]: dispatch 2024-03-21T13:04:30.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:29 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-109", "overlaypool": "test-rados-api-smithi043-81880-109-cache"}]: dispatch 2024-03-21T13:04:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:30 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-109", "overlaypool": "test-rados-api-smithi043-81880-109-cache"}]': finished 2024-03-21T13:04:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:30 smithi043 ceph-mon[30062]: osdmap e549: 8 total, 8 up, 8 in 2024-03-21T13:04:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:30 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-109-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:04:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:30 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-109-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:04:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:30 smithi043 ceph-mon[30062]: pgmap v847: 340 pgs: 340 active+clean; 582 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:30 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:04:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:30 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-109-cache", "mode": "writeback"}]': finished 2024-03-21T13:04:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:30 smithi043 ceph-mon[30062]: osdmap e550: 8 total, 8 up, 8 in 2024-03-21T13:04:31.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:30 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-109", "overlaypool": "test-rados-api-smithi043-81880-109-cache"}]': finished 2024-03-21T13:04:31.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:30 smithi043 ceph-mon[41699]: osdmap e549: 8 total, 8 up, 8 in 2024-03-21T13:04:31.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:30 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-109-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:04:31.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:30 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-109-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:04:31.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:30 smithi043 ceph-mon[41699]: pgmap v847: 340 pgs: 340 active+clean; 582 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:31.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:30 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:04:31.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:30 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-109-cache", "mode": "writeback"}]': finished 2024-03-21T13:04:31.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:30 smithi043 ceph-mon[41699]: osdmap e550: 8 total, 8 up, 8 in 2024-03-21T13:04:31.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:30 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-109", "overlaypool": "test-rados-api-smithi043-81880-109-cache"}]': finished 2024-03-21T13:04:31.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:30 smithi155 ceph-mon[33846]: osdmap e549: 8 total, 8 up, 8 in 2024-03-21T13:04:31.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:30 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-109-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:04:31.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:30 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-109-cache", "mode": "writeback"}]: dispatch 2024-03-21T13:04:31.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:30 smithi155 ceph-mon[33846]: pgmap v847: 340 pgs: 340 active+clean; 582 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:31.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:30 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:04:31.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:30 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-109-cache", "mode": "writeback"}]': finished 2024-03-21T13:04:31.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:30 smithi155 ceph-mon[33846]: osdmap e550: 8 total, 8 up, 8 in 2024-03-21T13:04:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:31 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:04:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:31 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:04:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:31 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:04:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:31 smithi043 ceph-mon[30062]: osdmap e551: 8 total, 8 up, 8 in 2024-03-21T13:04:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:31 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:04:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:31 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:04:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:31 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:04:32.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:31 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:04:32.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:31 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:04:32.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:31 smithi043 ceph-mon[41699]: osdmap e551: 8 total, 8 up, 8 in 2024-03-21T13:04:32.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:31 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:04:32.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:31 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:04:32.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:31 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:04:32.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:31 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:04:32.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:31 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:04:32.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:31 smithi155 ceph-mon[33846]: osdmap e551: 8 total, 8 up, 8 in 2024-03-21T13:04:32.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:31 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:04:32.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:31 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:04:33.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:32 smithi155 ceph-mon[33846]: pgmap v850: 340 pgs: 340 active+clean; 582 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:33.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:32 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:04:33.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:32 smithi155 ceph-mon[33846]: osdmap e552: 8 total, 8 up, 8 in 2024-03-21T13:04:33.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:32 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:04:33.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:32 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:04:33.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:32 smithi043 ceph-mon[30062]: pgmap v850: 340 pgs: 340 active+clean; 582 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:33.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:32 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:04:33.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:32 smithi043 ceph-mon[30062]: osdmap e552: 8 total, 8 up, 8 in 2024-03-21T13:04:33.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:32 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:04:33.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:32 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:04:33.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:32 smithi043 ceph-mon[41699]: pgmap v850: 340 pgs: 340 active+clean; 582 KiB data, 604 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:33.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:32 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:04:33.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:32 smithi043 ceph-mon[41699]: osdmap e552: 8 total, 8 up, 8 in 2024-03-21T13:04:33.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:32 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:04:33.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:32 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:04:34.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:33 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:04:34.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:33 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-21T13:04:34.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:33 smithi155 ceph-mon[33846]: osdmap e553: 8 total, 8 up, 8 in 2024-03-21T13:04:34.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:33 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-21T13:04:34.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:33 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-21T13:04:34.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:33 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:04:34.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:33 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-21T13:04:34.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:33 smithi043 ceph-mon[30062]: osdmap e553: 8 total, 8 up, 8 in 2024-03-21T13:04:34.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:33 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-21T13:04:34.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:33 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-21T13:04:34.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:33 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:04:34.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:33 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-21T13:04:34.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:33 smithi043 ceph-mon[41699]: osdmap e553: 8 total, 8 up, 8 in 2024-03-21T13:04:34.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:33 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-21T13:04:34.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:33 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-21T13:04:35.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:35 smithi155 ceph-mon[33846]: pgmap v853: 340 pgs: 340 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:35.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:35 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:04:35.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:04:35.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:35 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-03-21T13:04:35.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:35 smithi155 ceph-mon[33846]: osdmap e554: 8 total, 8 up, 8 in 2024-03-21T13:04:35.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:35 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:35.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:35 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:35 smithi043 ceph-mon[30062]: pgmap v853: 340 pgs: 340 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:35 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:04:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:04:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:35 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-03-21T13:04:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:35 smithi043 ceph-mon[30062]: osdmap e554: 8 total, 8 up, 8 in 2024-03-21T13:04:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:35 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:35 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:35.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:35 smithi043 ceph-mon[41699]: pgmap v853: 340 pgs: 340 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:35.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:35 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:04:35.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:04:35.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:35 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-03-21T13:04:35.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:35 smithi043 ceph-mon[41699]: osdmap e554: 8 total, 8 up, 8 in 2024-03-21T13:04:35.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:35 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:35.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:35 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:37.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:36 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-109"}]': finished 2024-03-21T13:04:37.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:36 smithi155 ceph-mon[33846]: osdmap e555: 8 total, 8 up, 8 in 2024-03-21T13:04:37.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-109", "tierpool": "test-rados-api-smithi043-81880-109-cache"}]: dispatch 2024-03-21T13:04:37.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:36 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-109", "tierpool": "test-rados-api-smithi043-81880-109-cache"}]: dispatch 2024-03-21T13:04:37.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:36 smithi155 ceph-mon[33846]: pgmap v856: 340 pgs: 340 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:37.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:36 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-109"}]': finished 2024-03-21T13:04:37.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:36 smithi043 ceph-mon[30062]: osdmap e555: 8 total, 8 up, 8 in 2024-03-21T13:04:37.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-109", "tierpool": "test-rados-api-smithi043-81880-109-cache"}]: dispatch 2024-03-21T13:04:37.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:36 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-109", "tierpool": "test-rados-api-smithi043-81880-109-cache"}]: dispatch 2024-03-21T13:04:37.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:36 smithi043 ceph-mon[30062]: pgmap v856: 340 pgs: 340 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:37.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:36 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-109"}]': finished 2024-03-21T13:04:37.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:36 smithi043 ceph-mon[41699]: osdmap e555: 8 total, 8 up, 8 in 2024-03-21T13:04:37.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-109", "tierpool": "test-rados-api-smithi043-81880-109-cache"}]: dispatch 2024-03-21T13:04:37.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:36 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-109", "tierpool": "test-rados-api-smithi043-81880-109-cache"}]: dispatch 2024-03-21T13:04:37.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:36 smithi043 ceph-mon[41699]: pgmap v856: 340 pgs: 340 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:38.386 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:38 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-109", "tierpool": "test-rados-api-smithi043-81880-109-cache"}]': finished 2024-03-21T13:04:38.386 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:38 smithi043 ceph-mon[30062]: osdmap e556: 8 total, 8 up, 8 in 2024-03-21T13:04:38.386 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:04:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:04:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:04:38.387 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:38 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-109", "tierpool": "test-rados-api-smithi043-81880-109-cache"}]': finished 2024-03-21T13:04:38.387 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:38 smithi043 ceph-mon[41699]: osdmap e556: 8 total, 8 up, 8 in 2024-03-21T13:04:38.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:38 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-109", "tierpool": "test-rados-api-smithi043-81880-109-cache"}]': finished 2024-03-21T13:04:38.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:38 smithi155 ceph-mon[33846]: osdmap e556: 8 total, 8 up, 8 in 2024-03-21T13:04:39.010 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:04:38 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:04:39.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:39 smithi155 ceph-mon[33846]: osdmap e557: 8 total, 8 up, 8 in 2024-03-21T13:04:39.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:39 smithi155 ceph-mon[33846]: pgmap v859: 308 pgs: 308 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:04:39.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:39 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:04:39.483 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:39 smithi043 ceph-mon[41699]: osdmap e557: 8 total, 8 up, 8 in 2024-03-21T13:04:39.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:39 smithi043 ceph-mon[41699]: pgmap v859: 308 pgs: 308 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:04:39.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:39 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:04:39.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:39 smithi043 ceph-mon[30062]: osdmap e557: 8 total, 8 up, 8 in 2024-03-21T13:04:39.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:39 smithi043 ceph-mon[30062]: pgmap v859: 308 pgs: 308 active+clean; 582 KiB data, 612 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:04:39.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:39 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:04:40.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:40 smithi155 ceph-mon[33846]: osdmap e558: 8 total, 8 up, 8 in 2024-03-21T13:04:40.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:40 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:40.459 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:40 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:40.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:40 smithi043 ceph-mon[30062]: osdmap e558: 8 total, 8 up, 8 in 2024-03-21T13:04:40.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:40 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:40.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:40 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:40.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:40 smithi043 ceph-mon[41699]: osdmap e558: 8 total, 8 up, 8 in 2024-03-21T13:04:40.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:40 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:40.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:40 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:41.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:41 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-109"}]': finished 2024-03-21T13:04:41.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:41 smithi155 ceph-mon[33846]: osdmap e559: 8 total, 8 up, 8 in 2024-03-21T13:04:41.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:41.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:41 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:41.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:41 smithi155 ceph-mon[33846]: pgmap v862: 300 pgs: 300 active+clean; 582 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:41.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:41 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-109"}]': finished 2024-03-21T13:04:41.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:41 smithi043 ceph-mon[30062]: osdmap e559: 8 total, 8 up, 8 in 2024-03-21T13:04:41.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:41.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:41 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:41.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:41 smithi043 ceph-mon[30062]: pgmap v862: 300 pgs: 300 active+clean; 582 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:41.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:41 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-109"}]': finished 2024-03-21T13:04:41.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:41 smithi043 ceph-mon[41699]: osdmap e559: 8 total, 8 up, 8 in 2024-03-21T13:04:41.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4105287808' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:41.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:41 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-109"}]: dispatch 2024-03-21T13:04:41.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:41 smithi043 ceph-mon[41699]: pgmap v862: 300 pgs: 300 active+clean; 582 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:42.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:42 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:04:42.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:42 smithi155 ceph-mon[33846]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-109"}]': finished 2024-03-21T13:04:42.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:42 smithi155 ceph-mon[33846]: osdmap e560: 8 total, 8 up, 8 in 2024-03-21T13:04:42.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:42 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:04:42.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:42 smithi043 ceph-mon[30062]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-109"}]': finished 2024-03-21T13:04:42.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:42 smithi043 ceph-mon[30062]: osdmap e560: 8 total, 8 up, 8 in 2024-03-21T13:04:42.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:42 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:04:42.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:42 smithi043 ceph-mon[41699]: from='client.26344 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-109"}]': finished 2024-03-21T13:04:42.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:42 smithi043 ceph-mon[41699]: osdmap e560: 8 total, 8 up, 8 in 2024-03-21T13:04:43.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:43 smithi155 ceph-mon[33846]: osdmap e561: 8 total, 8 up, 8 in 2024-03-21T13:04:43.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3927405983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:43.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:43 smithi155 ceph-mon[33846]: from='client.36338 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:43.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:43 smithi155 ceph-mon[33846]: pgmap v865: 292 pgs: 292 active+clean; 582 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-21T13:04:43.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:43 smithi043 ceph-mon[30062]: osdmap e561: 8 total, 8 up, 8 in 2024-03-21T13:04:43.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3927405983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:43.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:43 smithi043 ceph-mon[30062]: from='client.36338 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:43.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:43 smithi043 ceph-mon[30062]: pgmap v865: 292 pgs: 292 active+clean; 582 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-21T13:04:43.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:43 smithi043 ceph-mon[41699]: osdmap e561: 8 total, 8 up, 8 in 2024-03-21T13:04:43.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3927405983' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:43.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:43 smithi043 ceph-mon[41699]: from='client.36338 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:43.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:43 smithi043 ceph-mon[41699]: pgmap v865: 292 pgs: 292 active+clean; 582 KiB data, 613 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-21T13:04:44.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:44 smithi155 ceph-mon[33846]: from='client.36338 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-81880-104"}]': finished 2024-03-21T13:04:44.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:44 smithi155 ceph-mon[33846]: osdmap e562: 8 total, 8 up, 8 in 2024-03-21T13:04:44.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:44 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3927405983' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:44.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:44 smithi155 ceph-mon[33846]: from='client.36338 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:44.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:44 smithi043 ceph-mon[30062]: from='client.36338 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-81880-104"}]': finished 2024-03-21T13:04:44.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:44 smithi043 ceph-mon[30062]: osdmap e562: 8 total, 8 up, 8 in 2024-03-21T13:04:44.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:44 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3927405983' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:44.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:44 smithi043 ceph-mon[30062]: from='client.36338 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:44.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:44 smithi043 ceph-mon[41699]: from='client.36338 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi043-81880-104"}]': finished 2024-03-21T13:04:44.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:44 smithi043 ceph-mon[41699]: osdmap e562: 8 total, 8 up, 8 in 2024-03-21T13:04:44.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:44 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3927405983' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:44.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:44 smithi043 ceph-mon[41699]: from='client.36338 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-81880-104"}]: dispatch 2024-03-21T13:04:45.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:45 smithi155 ceph-mon[33846]: from='client.36338 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-81880-104"}]': finished 2024-03-21T13:04:45.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:45 smithi155 ceph-mon[33846]: osdmap e563: 8 total, 8 up, 8 in 2024-03-21T13:04:45.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:45 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:04:45.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:45 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:04:45.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:45 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:04:45.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:45 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:04:45.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:45 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81880-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:04:45.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:45 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81880-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:04:45.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:45 smithi155 ceph-mon[33846]: pgmap v868: 292 pgs: 292 active+clean; 582 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-21T13:04:45.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:45 smithi043 ceph-mon[30062]: from='client.36338 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-81880-104"}]': finished 2024-03-21T13:04:45.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:45 smithi043 ceph-mon[30062]: osdmap e563: 8 total, 8 up, 8 in 2024-03-21T13:04:45.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:04:45.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:45 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:04:45.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:04:45.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:45 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:04:45.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81880-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:04:45.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:45 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81880-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:04:45.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:45 smithi043 ceph-mon[30062]: pgmap v868: 292 pgs: 292 active+clean; 582 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-21T13:04:45.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:45 smithi043 ceph-mon[41699]: from='client.36338 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi043-81880-104"}]': finished 2024-03-21T13:04:45.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:45 smithi043 ceph-mon[41699]: osdmap e563: 8 total, 8 up, 8 in 2024-03-21T13:04:45.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:45 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:04:45.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:45 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:04:45.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:45 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:04:45.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:45 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:04:45.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:45 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81880-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:04:45.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:45 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81880-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-21T13:04:45.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:45 smithi043 ceph-mon[41699]: pgmap v868: 292 pgs: 292 active+clean; 582 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-21T13:04:46.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:46 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81880-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:04:46.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:46 smithi155 ceph-mon[33846]: osdmap e564: 8 total, 8 up, 8 in 2024-03-21T13:04:46.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81880-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:04:46.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:46 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81880-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:04:46.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:46 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81880-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:04:46.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:46 smithi043 ceph-mon[30062]: osdmap e564: 8 total, 8 up, 8 in 2024-03-21T13:04:46.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81880-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:04:46.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:46 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81880-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:04:46.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:46 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi043-81880-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-21T13:04:46.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:46 smithi043 ceph-mon[41699]: osdmap e564: 8 total, 8 up, 8 in 2024-03-21T13:04:46.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81880-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:04:46.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:46 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81880-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:04:47.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:47 smithi155 ceph-mon[33846]: pgmap v871: 292 pgs: 292 active+clean; 582 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:47.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:47 smithi155 ceph-mon[33846]: osdmap e565: 8 total, 8 up, 8 in 2024-03-21T13:04:47.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:47 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81880-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:04:47.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:47 smithi155 ceph-mon[33846]: osdmap e566: 8 total, 8 up, 8 in 2024-03-21T13:04:47.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:47 smithi043 ceph-mon[30062]: pgmap v871: 292 pgs: 292 active+clean; 582 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:47.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:47 smithi043 ceph-mon[30062]: osdmap e565: 8 total, 8 up, 8 in 2024-03-21T13:04:47.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:47 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81880-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:04:47.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:47 smithi043 ceph-mon[30062]: osdmap e566: 8 total, 8 up, 8 in 2024-03-21T13:04:47.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:47 smithi043 ceph-mon[41699]: pgmap v871: 292 pgs: 292 active+clean; 582 KiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:47.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:47 smithi043 ceph-mon[41699]: osdmap e565: 8 total, 8 up, 8 in 2024-03-21T13:04:47.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:47 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi043-81880-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:04:47.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:47 smithi043 ceph-mon[41699]: osdmap e566: 8 total, 8 up, 8 in 2024-03-21T13:04:48.318 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:04:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:04:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:04:48.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:48 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:04:48.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:48 smithi155 ceph-mon[33846]: osdmap e567: 8 total, 8 up, 8 in 2024-03-21T13:04:48.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:48 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:48.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:48 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:48.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:48 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:04:48.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:48 smithi043 ceph-mon[30062]: osdmap e567: 8 total, 8 up, 8 in 2024-03-21T13:04:48.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:48 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:48.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:48 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:48.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:48 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:04:48.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:48 smithi043 ceph-mon[41699]: osdmap e567: 8 total, 8 up, 8 in 2024-03-21T13:04:48.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:48 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:48.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:48 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:49.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:04:48 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:04:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:49 smithi155 ceph-mon[33846]: pgmap v874: 332 pgs: 40 unknown, 292 active+clean; 582 KiB data, 614 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:04:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:49 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:04:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:49 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:04:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:49 smithi155 ceph-mon[33846]: osdmap e568: 8 total, 8 up, 8 in 2024-03-21T13:04:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:49 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:04:49.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:49 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:04:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:49 smithi043 ceph-mon[30062]: pgmap v874: 332 pgs: 40 unknown, 292 active+clean; 582 KiB data, 614 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:04:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:49 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:04:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:49 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:04:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:49 smithi043 ceph-mon[30062]: osdmap e568: 8 total, 8 up, 8 in 2024-03-21T13:04:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:49 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:04:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:49 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:04:49.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:49 smithi043 ceph-mon[41699]: pgmap v874: 332 pgs: 40 unknown, 292 active+clean; 582 KiB data, 614 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:04:49.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:49 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:04:49.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:49 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:04:49.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:49 smithi043 ceph-mon[41699]: osdmap e568: 8 total, 8 up, 8 in 2024-03-21T13:04:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:49 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:04:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:49 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:04:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:50 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:04:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:50 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-112", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:04:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:50 smithi155 ceph-mon[33846]: osdmap e569: 8 total, 8 up, 8 in 2024-03-21T13:04:50.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:50 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-112"}]: dispatch 2024-03-21T13:04:50.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:50 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-112"}]: dispatch 2024-03-21T13:04:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:50 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:04:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:50 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-112", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:04:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:50 smithi043 ceph-mon[41699]: osdmap e569: 8 total, 8 up, 8 in 2024-03-21T13:04:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:50 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-112"}]: dispatch 2024-03-21T13:04:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:50 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-112"}]: dispatch 2024-03-21T13:04:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:50 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:04:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:50 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-112", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:04:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:50 smithi043 ceph-mon[30062]: osdmap e569: 8 total, 8 up, 8 in 2024-03-21T13:04:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:50 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-112"}]: dispatch 2024-03-21T13:04:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:50 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-112"}]: dispatch 2024-03-21T13:04:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:51 smithi155 ceph-mon[33846]: pgmap v877: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:51 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-112"}]': finished 2024-03-21T13:04:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:51 smithi155 ceph-mon[33846]: osdmap e570: 8 total, 8 up, 8 in 2024-03-21T13:04:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:51 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:04:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:51 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:04:51.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:51 smithi043 ceph-mon[30062]: pgmap v877: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:51.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:51 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-112"}]': finished 2024-03-21T13:04:51.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:51 smithi043 ceph-mon[30062]: osdmap e570: 8 total, 8 up, 8 in 2024-03-21T13:04:51.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:51 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:04:51.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:51 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:04:51.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:51 smithi043 ceph-mon[41699]: pgmap v877: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:51.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:51 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-112"}]': finished 2024-03-21T13:04:51.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:51 smithi043 ceph-mon[41699]: osdmap e570: 8 total, 8 up, 8 in 2024-03-21T13:04:51.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:51 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:04:51.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:51 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:04:53.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:53 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:04:53.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:53 smithi155 ceph-mon[33846]: osdmap e571: 8 total, 8 up, 8 in 2024-03-21T13:04:53.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:53 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-112"}]: dispatch 2024-03-21T13:04:53.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:53 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-112"}]: dispatch 2024-03-21T13:04:53.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:53 smithi155 ceph-mon[33846]: pgmap v880: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:53.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:53 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:04:53.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:53 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:04:53.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:53 smithi043 ceph-mon[41699]: osdmap e571: 8 total, 8 up, 8 in 2024-03-21T13:04:53.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:53 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-112"}]: dispatch 2024-03-21T13:04:53.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:53 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-112"}]: dispatch 2024-03-21T13:04:53.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:53 smithi043 ceph-mon[41699]: pgmap v880: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:53.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:53 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:04:53.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:53 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:04:53.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:53 smithi043 ceph-mon[30062]: osdmap e571: 8 total, 8 up, 8 in 2024-03-21T13:04:53.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:53 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-112"}]: dispatch 2024-03-21T13:04:53.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:53 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-112"}]: dispatch 2024-03-21T13:04:53.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:53 smithi043 ceph-mon[30062]: pgmap v880: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:04:53.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:53 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:04:54.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:54 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-112"}]': finished 2024-03-21T13:04:54.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:54 smithi155 ceph-mon[33846]: osdmap e572: 8 total, 8 up, 8 in 2024-03-21T13:04:54.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:54 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-112"}]': finished 2024-03-21T13:04:54.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:54 smithi043 ceph-mon[30062]: osdmap e572: 8 total, 8 up, 8 in 2024-03-21T13:04:54.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:54 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-112"}]': finished 2024-03-21T13:04:54.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:54 smithi043 ceph-mon[41699]: osdmap e572: 8 total, 8 up, 8 in 2024-03-21T13:04:55.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:55 smithi155 ceph-mon[33846]: osdmap e573: 8 total, 8 up, 8 in 2024-03-21T13:04:55.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:55 smithi155 ceph-mon[33846]: pgmap v883: 300 pgs: 300 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:04:55.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:55 smithi043 ceph-mon[30062]: osdmap e573: 8 total, 8 up, 8 in 2024-03-21T13:04:55.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:55 smithi043 ceph-mon[30062]: pgmap v883: 300 pgs: 300 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:04:55.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:55 smithi043 ceph-mon[41699]: osdmap e573: 8 total, 8 up, 8 in 2024-03-21T13:04:55.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:55 smithi043 ceph-mon[41699]: pgmap v883: 300 pgs: 300 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:04:56.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:56 smithi155 ceph-mon[33846]: osdmap e574: 8 total, 8 up, 8 in 2024-03-21T13:04:56.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:56 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:56.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:56 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:56.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:56 smithi043 ceph-mon[30062]: osdmap e574: 8 total, 8 up, 8 in 2024-03-21T13:04:56.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:56 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:56.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:56 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:56.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:56 smithi043 ceph-mon[41699]: osdmap e574: 8 total, 8 up, 8 in 2024-03-21T13:04:56.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:56 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:56.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:56 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:04:57.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:57 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:04:57.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:57 smithi155 ceph-mon[33846]: osdmap e575: 8 total, 8 up, 8 in 2024-03-21T13:04:57.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:57 smithi155 ceph-mon[33846]: pgmap v886: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:04:57.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:57 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:04:57.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:57 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:04:57.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:57 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:04:57.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:57 smithi043 ceph-mon[30062]: osdmap e575: 8 total, 8 up, 8 in 2024-03-21T13:04:57.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:57 smithi043 ceph-mon[30062]: pgmap v886: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:04:57.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:57 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:04:57.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:57 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:04:57.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:57 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:04:57.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:57 smithi043 ceph-mon[41699]: osdmap e575: 8 total, 8 up, 8 in 2024-03-21T13:04:57.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:57 smithi043 ceph-mon[41699]: pgmap v886: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:04:57.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:57 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:04:57.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:57 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:04:58.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:58 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-114", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:04:58.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:58 smithi155 ceph-mon[33846]: osdmap e576: 8 total, 8 up, 8 in 2024-03-21T13:04:58.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:58 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-114"}]: dispatch 2024-03-21T13:04:58.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:58 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-114"}]: dispatch 2024-03-21T13:04:58.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:04:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:04:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:04:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:58 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-114", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:04:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:58 smithi043 ceph-mon[30062]: osdmap e576: 8 total, 8 up, 8 in 2024-03-21T13:04:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:58 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-114"}]: dispatch 2024-03-21T13:04:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:58 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-114"}]: dispatch 2024-03-21T13:04:58.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:58 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-114", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:04:58.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:58 smithi043 ceph-mon[41699]: osdmap e576: 8 total, 8 up, 8 in 2024-03-21T13:04:58.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:58 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-114"}]: dispatch 2024-03-21T13:04:58.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:58 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-114"}]: dispatch 2024-03-21T13:04:59.119 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:04:58 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:04:59.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:59 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-114"}]': finished 2024-03-21T13:04:59.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:59 smithi155 ceph-mon[33846]: osdmap e577: 8 total, 8 up, 8 in 2024-03-21T13:04:59.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:59 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-114", "mode": "writeback"}]: dispatch 2024-03-21T13:04:59.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:59 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-114", "mode": "writeback"}]: dispatch 2024-03-21T13:04:59.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:59 smithi155 ceph-mon[33846]: pgmap v889: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:04:59.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:59 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:04:59.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:59 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:04:59.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:59 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-114", "mode": "writeback"}]': finished 2024-03-21T13:04:59.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:04:59 smithi155 ceph-mon[33846]: osdmap e578: 8 total, 8 up, 8 in 2024-03-21T13:04:59.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:59 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-114"}]': finished 2024-03-21T13:04:59.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:59 smithi043 ceph-mon[30062]: osdmap e577: 8 total, 8 up, 8 in 2024-03-21T13:04:59.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:59 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-114", "mode": "writeback"}]: dispatch 2024-03-21T13:04:59.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:59 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-114", "mode": "writeback"}]: dispatch 2024-03-21T13:04:59.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:59 smithi043 ceph-mon[30062]: pgmap v889: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:04:59.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:59 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:04:59.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:59 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:04:59.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:59 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-114", "mode": "writeback"}]': finished 2024-03-21T13:04:59.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:04:59 smithi043 ceph-mon[30062]: osdmap e578: 8 total, 8 up, 8 in 2024-03-21T13:04:59.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:59 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-114"}]': finished 2024-03-21T13:04:59.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:59 smithi043 ceph-mon[41699]: osdmap e577: 8 total, 8 up, 8 in 2024-03-21T13:04:59.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:59 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-114", "mode": "writeback"}]: dispatch 2024-03-21T13:04:59.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:59 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-114", "mode": "writeback"}]: dispatch 2024-03-21T13:04:59.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:59 smithi043 ceph-mon[41699]: pgmap v889: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:04:59.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:59 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:04:59.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:59 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:04:59.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:59 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-114", "mode": "writeback"}]': finished 2024-03-21T13:04:59.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:04:59 smithi043 ceph-mon[41699]: osdmap e578: 8 total, 8 up, 8 in 2024-03-21T13:05:00.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:00 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:00.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:00 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:00.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:00 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:00.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:00 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:00.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:00 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:00.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:00 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:01.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:01 smithi155 ceph-mon[33846]: pgmap v891: 332 pgs: 332 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-03-21T13:05:01.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:01 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:05:01.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:01 smithi155 ceph-mon[33846]: osdmap e579: 8 total, 8 up, 8 in 2024-03-21T13:05:01.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:01 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-114"}]: dispatch 2024-03-21T13:05:01.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:01 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-114"}]: dispatch 2024-03-21T13:05:01.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:01 smithi043 ceph-mon[30062]: pgmap v891: 332 pgs: 332 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-03-21T13:05:01.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:01 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:05:01.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:01 smithi043 ceph-mon[30062]: osdmap e579: 8 total, 8 up, 8 in 2024-03-21T13:05:01.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:01 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-114"}]: dispatch 2024-03-21T13:05:01.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:01 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-114"}]: dispatch 2024-03-21T13:05:01.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:01 smithi043 ceph-mon[41699]: pgmap v891: 332 pgs: 332 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 254 B/s wr, 1 op/s 2024-03-21T13:05:01.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:01 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:05:01.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:01 smithi043 ceph-mon[41699]: osdmap e579: 8 total, 8 up, 8 in 2024-03-21T13:05:01.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:01 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-114"}]: dispatch 2024-03-21T13:05:01.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:01 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-114"}]: dispatch 2024-03-21T13:05:02.332 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:02 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:05:02.332 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:02 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-114"}]': finished 2024-03-21T13:05:02.332 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:02 smithi043 ceph-mon[30062]: osdmap e580: 8 total, 8 up, 8 in 2024-03-21T13:05:02.332 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:02 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:05:02.332 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:02 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:05:02.333 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:02 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-114"}]': finished 2024-03-21T13:05:02.333 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:02 smithi043 ceph-mon[41699]: osdmap e580: 8 total, 8 up, 8 in 2024-03-21T13:05:02.333 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:02 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:05:02.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:02 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:05:02.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:02 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-114"}]': finished 2024-03-21T13:05:02.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:02 smithi155 ceph-mon[33846]: osdmap e580: 8 total, 8 up, 8 in 2024-03-21T13:05:02.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:02 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:05:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:03 smithi155 ceph-mon[33846]: pgmap v894: 332 pgs: 332 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:05:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:03 smithi155 ceph-mon[33846]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:05:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:03 smithi155 ceph-mon[33846]: osdmap e581: 8 total, 8 up, 8 in 2024-03-21T13:05:03.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:03 smithi043 ceph-mon[30062]: pgmap v894: 332 pgs: 332 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:05:03.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:03 smithi043 ceph-mon[30062]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:05:03.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:03 smithi043 ceph-mon[30062]: osdmap e581: 8 total, 8 up, 8 in 2024-03-21T13:05:03.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:03 smithi043 ceph-mon[41699]: pgmap v894: 332 pgs: 332 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:05:03.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:03 smithi043 ceph-mon[41699]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:05:03.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:03 smithi043 ceph-mon[41699]: osdmap e581: 8 total, 8 up, 8 in 2024-03-21T13:05:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:04 smithi043 ceph-mon[30062]: osdmap e582: 8 total, 8 up, 8 in 2024-03-21T13:05:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:04 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:04 smithi043 ceph-mon[41699]: osdmap e582: 8 total, 8 up, 8 in 2024-03-21T13:05:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:04 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:04.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:04 smithi155 ceph-mon[33846]: osdmap e582: 8 total, 8 up, 8 in 2024-03-21T13:05:04.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:04.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:04 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:05.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:05 smithi155 ceph-mon[33846]: pgmap v897: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:05:05.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:05 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:05:05.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:05 smithi155 ceph-mon[33846]: osdmap e583: 8 total, 8 up, 8 in 2024-03-21T13:05:05.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:05 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:05:05.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:05 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:05:05.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:05 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:05:05.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:05 smithi043 ceph-mon[30062]: pgmap v897: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:05:05.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:05 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:05:05.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:05 smithi043 ceph-mon[30062]: osdmap e583: 8 total, 8 up, 8 in 2024-03-21T13:05:05.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:05 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:05:05.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:05 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:05:05.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:05 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:05:05.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:05 smithi043 ceph-mon[41699]: pgmap v897: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:05:05.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:05 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:05:05.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:05 smithi043 ceph-mon[41699]: osdmap e583: 8 total, 8 up, 8 in 2024-03-21T13:05:05.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:05 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:05:05.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:05 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:05:05.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:05 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:05:06.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:06 smithi155 ceph-mon[33846]: osdmap e584: 8 total, 8 up, 8 in 2024-03-21T13:05:06.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:06 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:06.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:06 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:06.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:06 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:05:06.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:06 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:05:06.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:06 smithi043 ceph-mon[30062]: osdmap e584: 8 total, 8 up, 8 in 2024-03-21T13:05:06.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:06 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:06.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:06 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:06.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:06 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:05:06.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:06 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:05:06.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:06 smithi043 ceph-mon[41699]: osdmap e584: 8 total, 8 up, 8 in 2024-03-21T13:05:06.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:06 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:06.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:06 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:06.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:06 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:05:06.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:06 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:05:07.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:07 smithi155 ceph-mon[33846]: pgmap v900: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:05:07.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:07 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-116", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:05:07.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:07 smithi155 ceph-mon[33846]: osdmap e585: 8 total, 8 up, 8 in 2024-03-21T13:05:07.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:07 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-116"}]: dispatch 2024-03-21T13:05:07.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:07 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-116"}]: dispatch 2024-03-21T13:05:07.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:07 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:05:07.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:07 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:05:07.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:07 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:05:07.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:07 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-116"}]': finished 2024-03-21T13:05:07.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:07 smithi043 ceph-mon[30062]: pgmap v900: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:05:07.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:07 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-116", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:05:07.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:07 smithi043 ceph-mon[30062]: osdmap e585: 8 total, 8 up, 8 in 2024-03-21T13:05:07.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:07 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-116"}]: dispatch 2024-03-21T13:05:07.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:07 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-116"}]: dispatch 2024-03-21T13:05:07.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:07 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:05:07.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:07 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:05:07.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:07 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:05:07.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:07 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-116"}]': finished 2024-03-21T13:05:07.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:07 smithi043 ceph-mon[41699]: pgmap v900: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:05:07.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:07 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-116", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:05:07.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:07 smithi043 ceph-mon[41699]: osdmap e585: 8 total, 8 up, 8 in 2024-03-21T13:05:07.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:07 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-116"}]: dispatch 2024-03-21T13:05:07.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:07 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-116"}]: dispatch 2024-03-21T13:05:07.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:07 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:05:07.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:07 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:05:07.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:07 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:05:07.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:07 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-116"}]': finished 2024-03-21T13:05:08.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:08 smithi043 ceph-mon[30062]: osdmap e586: 8 total, 8 up, 8 in 2024-03-21T13:05:08.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:08 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-116", "mode": "writeback"}]: dispatch 2024-03-21T13:05:08.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:08 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-116", "mode": "writeback"}]: dispatch 2024-03-21T13:05:08.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:08 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:05:08.386 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:05:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:05:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:05:08.386 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:08 smithi043 ceph-mon[41699]: osdmap e586: 8 total, 8 up, 8 in 2024-03-21T13:05:08.386 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:08 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-116", "mode": "writeback"}]: dispatch 2024-03-21T13:05:08.386 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:08 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-116", "mode": "writeback"}]: dispatch 2024-03-21T13:05:08.386 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:08 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:05:08.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:08 smithi155 ceph-mon[33846]: osdmap e586: 8 total, 8 up, 8 in 2024-03-21T13:05:08.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:08 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-116", "mode": "writeback"}]: dispatch 2024-03-21T13:05:08.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:08 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-116", "mode": "writeback"}]: dispatch 2024-03-21T13:05:08.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:08 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:05:09.171 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:05:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:05:09.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:09 smithi155 ceph-mon[33846]: pgmap v903: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:05:09.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:09 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:05:09.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:09 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-116", "mode": "writeback"}]': finished 2024-03-21T13:05:09.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:09 smithi155 ceph-mon[33846]: osdmap e587: 8 total, 8 up, 8 in 2024-03-21T13:05:09.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:09 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "314.3"}]: dispatch 2024-03-21T13:05:09.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:09 smithi155 ceph-mon[33846]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "314.3"}]: dispatch 2024-03-21T13:05:09.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:09 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:05:09.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:09 smithi043 ceph-mon[30062]: pgmap v903: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:05:09.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:09 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:05:09.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:09 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-116", "mode": "writeback"}]': finished 2024-03-21T13:05:09.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:09 smithi043 ceph-mon[30062]: osdmap e587: 8 total, 8 up, 8 in 2024-03-21T13:05:09.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:09 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "314.3"}]: dispatch 2024-03-21T13:05:09.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:09 smithi043 ceph-mon[30062]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "314.3"}]: dispatch 2024-03-21T13:05:09.486 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:09 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:05:09.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:09 smithi043 ceph-mon[41699]: pgmap v903: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:05:09.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:09 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:05:09.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:09 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-116", "mode": "writeback"}]': finished 2024-03-21T13:05:09.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:09 smithi043 ceph-mon[41699]: osdmap e587: 8 total, 8 up, 8 in 2024-03-21T13:05:09.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:09 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "314.3"}]: dispatch 2024-03-21T13:05:09.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:09 smithi043 ceph-mon[41699]: from='mon.? v1:172.21.15.43:0/1636189224' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "314.3"}]: dispatch 2024-03-21T13:05:09.487 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:09 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:05:10.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:10 smithi155 ceph-mon[33846]: 314.3 scrub starts 2024-03-21T13:05:10.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:10 smithi155 ceph-mon[33846]: 314.3 scrub ok 2024-03-21T13:05:10.483 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:10 smithi043 ceph-mon[41699]: 314.3 scrub starts 2024-03-21T13:05:10.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:10 smithi043 ceph-mon[41699]: 314.3 scrub ok 2024-03-21T13:05:10.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:10 smithi043 ceph-mon[30062]: 314.3 scrub starts 2024-03-21T13:05:10.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:10 smithi043 ceph-mon[30062]: 314.3 scrub ok 2024-03-21T13:05:11.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:11 smithi155 ceph-mon[33846]: pgmap v905: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.4 KiB/s wr, 2 op/s 2024-03-21T13:05:11.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:11 smithi043 ceph-mon[30062]: pgmap v905: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.4 KiB/s wr, 2 op/s 2024-03-21T13:05:11.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:11 smithi043 ceph-mon[41699]: pgmap v905: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.4 KiB/s wr, 2 op/s 2024-03-21T13:05:13.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:13 smithi155 ceph-mon[33846]: pgmap v906: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:05:13.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:13 smithi043 ceph-mon[30062]: pgmap v906: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:05:13.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:13 smithi043 ceph-mon[41699]: pgmap v906: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:05:15.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:15 smithi155 ceph-mon[33846]: pgmap v907: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 899 B/s wr, 2 op/s 2024-03-21T13:05:15.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:15 smithi043 ceph-mon[30062]: pgmap v907: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 899 B/s wr, 2 op/s 2024-03-21T13:05:15.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:15 smithi043 ceph-mon[41699]: pgmap v907: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 899 B/s wr, 2 op/s 2024-03-21T13:05:17.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:17 smithi155 ceph-mon[33846]: pgmap v908: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 799 B/s wr, 2 op/s 2024-03-21T13:05:17.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:17 smithi043 ceph-mon[41699]: pgmap v908: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 799 B/s wr, 2 op/s 2024-03-21T13:05:17.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:17 smithi043 ceph-mon[30062]: pgmap v908: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 799 B/s wr, 2 op/s 2024-03-21T13:05:18.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:18 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "314.c", "id": [6, 0]}]: dispatch 2024-03-21T13:05:18.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:18 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "314.c", "id": [6, 0]}]: dispatch 2024-03-21T13:05:18.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:18 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "314.c", "id": [6, 0]}]: dispatch 2024-03-21T13:05:18.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:18 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "314.c", "id": [6, 0]}]: dispatch 2024-03-21T13:05:18.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:05:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:05:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:05:18.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:18 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "314.c", "id": [6, 0]}]: dispatch 2024-03-21T13:05:18.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:18 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "314.c", "id": [6, 0]}]: dispatch 2024-03-21T13:05:19.203 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:05:18 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:05:19.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:19 smithi155 ceph-mon[33846]: pgmap v909: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 716 B/s wr, 2 op/s 2024-03-21T13:05:19.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:19 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "314.c", "id": [6, 0]}]': finished 2024-03-21T13:05:19.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:19 smithi155 ceph-mon[33846]: osdmap e588: 8 total, 8 up, 8 in 2024-03-21T13:05:19.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:19 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:05:19.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:19 smithi043 ceph-mon[30062]: pgmap v909: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 716 B/s wr, 2 op/s 2024-03-21T13:05:19.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:19 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "314.c", "id": [6, 0]}]': finished 2024-03-21T13:05:19.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:19 smithi043 ceph-mon[30062]: osdmap e588: 8 total, 8 up, 8 in 2024-03-21T13:05:19.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:19 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:05:19.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:19 smithi043 ceph-mon[41699]: pgmap v909: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 716 B/s wr, 2 op/s 2024-03-21T13:05:19.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:19 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "314.c", "id": [6, 0]}]': finished 2024-03-21T13:05:19.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:19 smithi043 ceph-mon[41699]: osdmap e588: 8 total, 8 up, 8 in 2024-03-21T13:05:19.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:19 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:05:20.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:20 smithi155 ceph-mon[33846]: osdmap e589: 8 total, 8 up, 8 in 2024-03-21T13:05:20.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:20 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:05:20.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:20 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:05:20.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:20 smithi043 ceph-mon[30062]: osdmap e589: 8 total, 8 up, 8 in 2024-03-21T13:05:20.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:20 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:05:20.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:20 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:05:20.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:20 smithi043 ceph-mon[41699]: osdmap e589: 8 total, 8 up, 8 in 2024-03-21T13:05:20.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:20 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:05:20.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:20 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:05:21.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:21 smithi155 ceph-mon[33846]: pgmap v912: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-03-21T13:05:21.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:21 smithi043 ceph-mon[30062]: pgmap v912: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-03-21T13:05:21.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:21 smithi043 ceph-mon[41699]: pgmap v912: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-03-21T13:05:23.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:23 smithi155 ceph-mon[33846]: pgmap v913: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:05:23.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:23 smithi043 ceph-mon[30062]: pgmap v913: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:05:23.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:23 smithi043 ceph-mon[41699]: pgmap v913: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:05:24.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:24 smithi155 ceph-mon[33846]: osdmap e590: 8 total, 8 up, 8 in 2024-03-21T13:05:24.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:24 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:24.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:24 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:24.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:24 smithi043 ceph-mon[30062]: osdmap e590: 8 total, 8 up, 8 in 2024-03-21T13:05:24.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:24 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:24.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:24 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:24.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:24 smithi043 ceph-mon[41699]: osdmap e590: 8 total, 8 up, 8 in 2024-03-21T13:05:24.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:24 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:24.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:24 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:25.642 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.642 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.642 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.642 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.642 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.642 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.642 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.642 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.642 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.643 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.644 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.645 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.646 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.646 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.646 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.646 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.646 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.646 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.646 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.647 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.648 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.648 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.648 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.648 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.648 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.648 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.648 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.648 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.648 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.648 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.648 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.648 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.648 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.648 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.648 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.649 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:05:25.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:25 smithi155 ceph-mon[33846]: pgmap v915: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:05:25.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:25 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:05:25.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:25 smithi155 ceph-mon[33846]: osdmap e591: 8 total, 8 up, 8 in 2024-03-21T13:05:25.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:25 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-116"}]: dispatch 2024-03-21T13:05:25.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:25 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-116"}]: dispatch 2024-03-21T13:05:25.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:25 smithi043 ceph-mon[30062]: pgmap v915: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:05:25.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:25 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:05:25.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:25 smithi043 ceph-mon[30062]: osdmap e591: 8 total, 8 up, 8 in 2024-03-21T13:05:25.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:25 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-116"}]: dispatch 2024-03-21T13:05:25.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:25 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-116"}]: dispatch 2024-03-21T13:05:25.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:25 smithi043 ceph-mon[41699]: pgmap v915: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:05:25.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:25 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:05:25.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:25 smithi043 ceph-mon[41699]: osdmap e591: 8 total, 8 up, 8 in 2024-03-21T13:05:25.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:25 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-116"}]: dispatch 2024-03-21T13:05:25.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:25 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-116"}]: dispatch 2024-03-21T13:05:26.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:26 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:05:26.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:26 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-116"}]': finished 2024-03-21T13:05:26.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:26 smithi155 ceph-mon[33846]: osdmap e592: 8 total, 8 up, 8 in 2024-03-21T13:05:26.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:26 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:05:26.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:26 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-116"}]': finished 2024-03-21T13:05:26.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:26 smithi043 ceph-mon[30062]: osdmap e592: 8 total, 8 up, 8 in 2024-03-21T13:05:26.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:26 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:05:26.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:26 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-116"}]': finished 2024-03-21T13:05:26.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:26 smithi043 ceph-mon[41699]: osdmap e592: 8 total, 8 up, 8 in 2024-03-21T13:05:27.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:27 smithi155 ceph-mon[33846]: pgmap v918: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:05:27.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:27 smithi155 ceph-mon[33846]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:05:27.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:27 smithi155 ceph-mon[33846]: osdmap e593: 8 total, 8 up, 8 in 2024-03-21T13:05:27.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:27 smithi043 ceph-mon[30062]: pgmap v918: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:05:27.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:27 smithi043 ceph-mon[30062]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:05:27.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:27 smithi043 ceph-mon[30062]: osdmap e593: 8 total, 8 up, 8 in 2024-03-21T13:05:27.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:27 smithi043 ceph-mon[41699]: pgmap v918: 332 pgs: 332 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:05:27.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:27 smithi043 ceph-mon[41699]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:05:27.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:27 smithi043 ceph-mon[41699]: osdmap e593: 8 total, 8 up, 8 in 2024-03-21T13:05:28.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:05:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:05:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:05:28.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:05:28 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:05:28.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:28 smithi155 ceph-mon[33846]: osdmap e594: 8 total, 8 up, 8 in 2024-03-21T13:05:28.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:28.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:28 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:28.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:28 smithi043 ceph-mon[30062]: osdmap e594: 8 total, 8 up, 8 in 2024-03-21T13:05:28.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:28.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:28 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:28.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:28 smithi043 ceph-mon[41699]: osdmap e594: 8 total, 8 up, 8 in 2024-03-21T13:05:28.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:28.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:28 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:29 smithi155 ceph-mon[33846]: pgmap v921: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:05:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:29 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:05:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:29 smithi155 ceph-mon[33846]: osdmap e595: 8 total, 8 up, 8 in 2024-03-21T13:05:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:29 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:05:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:29 smithi043 ceph-mon[30062]: pgmap v921: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:05:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:29 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:05:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:29 smithi043 ceph-mon[30062]: osdmap e595: 8 total, 8 up, 8 in 2024-03-21T13:05:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:29 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:05:29.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:29 smithi043 ceph-mon[41699]: pgmap v921: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 629 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:05:29.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:29 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:05:29.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:29 smithi043 ceph-mon[41699]: osdmap e595: 8 total, 8 up, 8 in 2024-03-21T13:05:29.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:29 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:05:30.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:30 smithi155 ceph-mon[33846]: osdmap e596: 8 total, 8 up, 8 in 2024-03-21T13:05:30.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:30 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:30.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:30 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:30.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:30 smithi043 ceph-mon[30062]: osdmap e596: 8 total, 8 up, 8 in 2024-03-21T13:05:30.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:30 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:30.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:30 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:30.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:30 smithi043 ceph-mon[41699]: osdmap e596: 8 total, 8 up, 8 in 2024-03-21T13:05:30.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:30 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:30.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:30 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:31.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:31 smithi155 ceph-mon[33846]: pgmap v924: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-03-21T13:05:31.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:31 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-118", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:05:31.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:31 smithi155 ceph-mon[33846]: osdmap e597: 8 total, 8 up, 8 in 2024-03-21T13:05:31.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:31 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-118"}]: dispatch 2024-03-21T13:05:31.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:31 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-118"}]: dispatch 2024-03-21T13:05:31.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:31 smithi043 ceph-mon[30062]: pgmap v924: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-03-21T13:05:31.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:31 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-118", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:05:31.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:31 smithi043 ceph-mon[30062]: osdmap e597: 8 total, 8 up, 8 in 2024-03-21T13:05:31.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:31 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-118"}]: dispatch 2024-03-21T13:05:31.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:31 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-118"}]: dispatch 2024-03-21T13:05:31.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:31 smithi043 ceph-mon[41699]: pgmap v924: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-03-21T13:05:31.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:31 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-118", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:05:31.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:31 smithi043 ceph-mon[41699]: osdmap e597: 8 total, 8 up, 8 in 2024-03-21T13:05:31.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:31 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-118"}]: dispatch 2024-03-21T13:05:31.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:31 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-118"}]: dispatch 2024-03-21T13:05:32.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:32 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-118"}]': finished 2024-03-21T13:05:32.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:32 smithi155 ceph-mon[33846]: osdmap e598: 8 total, 8 up, 8 in 2024-03-21T13:05:32.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:32 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-118", "mode": "writeback"}]: dispatch 2024-03-21T13:05:32.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:32 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-118", "mode": "writeback"}]: dispatch 2024-03-21T13:05:32.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:32 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:05:32.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:32 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:05:32.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:32 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-118", "mode": "writeback"}]': finished 2024-03-21T13:05:32.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:32 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-118"}]': finished 2024-03-21T13:05:32.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:32 smithi043 ceph-mon[30062]: osdmap e598: 8 total, 8 up, 8 in 2024-03-21T13:05:32.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:32 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-118", "mode": "writeback"}]: dispatch 2024-03-21T13:05:32.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:32 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-118", "mode": "writeback"}]: dispatch 2024-03-21T13:05:32.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:32 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:05:32.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:32 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:05:32.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:32 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-118", "mode": "writeback"}]': finished 2024-03-21T13:05:32.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:32 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-118"}]': finished 2024-03-21T13:05:32.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:32 smithi043 ceph-mon[41699]: osdmap e598: 8 total, 8 up, 8 in 2024-03-21T13:05:32.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:32 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-118", "mode": "writeback"}]: dispatch 2024-03-21T13:05:32.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:32 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-118", "mode": "writeback"}]: dispatch 2024-03-21T13:05:32.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:32 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:05:32.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:32 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:05:32.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:32 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-118", "mode": "writeback"}]': finished 2024-03-21T13:05:33.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:33 smithi155 ceph-mon[33846]: pgmap v927: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:05:33.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:33 smithi155 ceph-mon[33846]: osdmap e599: 8 total, 8 up, 8 in 2024-03-21T13:05:33.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:33 smithi043 ceph-mon[30062]: pgmap v927: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:05:33.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:33 smithi043 ceph-mon[30062]: osdmap e599: 8 total, 8 up, 8 in 2024-03-21T13:05:33.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:33 smithi043 ceph-mon[41699]: pgmap v927: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 630 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:05:33.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:33 smithi043 ceph-mon[41699]: osdmap e599: 8 total, 8 up, 8 in 2024-03-21T13:05:34.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:34 smithi155 ceph-mon[33846]: osdmap e600: 8 total, 8 up, 8 in 2024-03-21T13:05:34.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:34 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:34.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:34 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:34.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:34 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:05:34.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:34 smithi043 ceph-mon[30062]: osdmap e600: 8 total, 8 up, 8 in 2024-03-21T13:05:34.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:34 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:34.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:34 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:34.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:34 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:05:34.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:34 smithi043 ceph-mon[41699]: osdmap e600: 8 total, 8 up, 8 in 2024-03-21T13:05:34.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:34 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:34.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:34 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:34.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:34 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:05:35.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:35 smithi155 ceph-mon[33846]: pgmap v930: 332 pgs: 332 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:05:35.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:35 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:05:35.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:35 smithi155 ceph-mon[33846]: osdmap e601: 8 total, 8 up, 8 in 2024-03-21T13:05:35.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:35 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-118"}]: dispatch 2024-03-21T13:05:35.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:35 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-118"}]: dispatch 2024-03-21T13:05:35.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:35 smithi043 ceph-mon[41699]: pgmap v930: 332 pgs: 332 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:05:35.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:35 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:05:35.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:35 smithi043 ceph-mon[41699]: osdmap e601: 8 total, 8 up, 8 in 2024-03-21T13:05:35.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:35 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-118"}]: dispatch 2024-03-21T13:05:35.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:35 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-118"}]: dispatch 2024-03-21T13:05:35.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:35 smithi043 ceph-mon[30062]: pgmap v930: 332 pgs: 332 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:05:35.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:35 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:05:35.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:35 smithi043 ceph-mon[30062]: osdmap e601: 8 total, 8 up, 8 in 2024-03-21T13:05:35.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:35 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-118"}]: dispatch 2024-03-21T13:05:35.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:35 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-118"}]: dispatch 2024-03-21T13:05:36.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:36 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:05:36.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:36 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-118"}]': finished 2024-03-21T13:05:36.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:36 smithi155 ceph-mon[33846]: osdmap e602: 8 total, 8 up, 8 in 2024-03-21T13:05:36.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:36 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:05:36.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:36 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-118"}]': finished 2024-03-21T13:05:36.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:36 smithi043 ceph-mon[30062]: osdmap e602: 8 total, 8 up, 8 in 2024-03-21T13:05:36.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:36 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:05:36.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:36 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-118"}]': finished 2024-03-21T13:05:36.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:36 smithi043 ceph-mon[41699]: osdmap e602: 8 total, 8 up, 8 in 2024-03-21T13:05:37.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:37 smithi155 ceph-mon[33846]: pgmap v933: 332 pgs: 332 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:05:37.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:37 smithi155 ceph-mon[33846]: osdmap e603: 8 total, 8 up, 8 in 2024-03-21T13:05:37.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:37 smithi155 ceph-mon[33846]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:05:37.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:37 smithi043 ceph-mon[30062]: pgmap v933: 332 pgs: 332 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:05:37.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:37 smithi043 ceph-mon[30062]: osdmap e603: 8 total, 8 up, 8 in 2024-03-21T13:05:37.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:37 smithi043 ceph-mon[30062]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:05:37.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:37 smithi043 ceph-mon[41699]: pgmap v933: 332 pgs: 332 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:05:37.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:37 smithi043 ceph-mon[41699]: osdmap e603: 8 total, 8 up, 8 in 2024-03-21T13:05:37.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:37 smithi043 ceph-mon[41699]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:05:38.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:05:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:05:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:05:38.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:05:38 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:05:38.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:38 smithi155 ceph-mon[33846]: osdmap e604: 8 total, 8 up, 8 in 2024-03-21T13:05:38.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:38 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:38.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:38 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:38.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:38 smithi043 ceph-mon[30062]: osdmap e604: 8 total, 8 up, 8 in 2024-03-21T13:05:38.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:38 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:38.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:38 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:38.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:38 smithi043 ceph-mon[41699]: osdmap e604: 8 total, 8 up, 8 in 2024-03-21T13:05:38.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:38 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:38.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:38 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:39.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:39 smithi155 ceph-mon[33846]: pgmap v936: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:05:39.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:39 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:05:39.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:39 smithi155 ceph-mon[33846]: osdmap e605: 8 total, 8 up, 8 in 2024-03-21T13:05:39.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:39 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:39.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:39 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:39.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:39 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:05:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:39 smithi043 ceph-mon[30062]: pgmap v936: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:05:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:39 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:05:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:39 smithi043 ceph-mon[30062]: osdmap e605: 8 total, 8 up, 8 in 2024-03-21T13:05:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:39 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:39 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:39 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:05:39.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:39 smithi043 ceph-mon[41699]: pgmap v936: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:05:39.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:39 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:05:39.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:39 smithi043 ceph-mon[41699]: osdmap e605: 8 total, 8 up, 8 in 2024-03-21T13:05:39.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:39 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:39.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:39 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:39.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:39 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:05:40.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:40 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-120", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:05:40.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:40 smithi043 ceph-mon[30062]: osdmap e606: 8 total, 8 up, 8 in 2024-03-21T13:05:40.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:40 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-120"}]: dispatch 2024-03-21T13:05:40.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:40 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-120"}]: dispatch 2024-03-21T13:05:40.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:40 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-120", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:05:40.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:40 smithi043 ceph-mon[41699]: osdmap e606: 8 total, 8 up, 8 in 2024-03-21T13:05:40.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:40 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-120"}]: dispatch 2024-03-21T13:05:40.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:40 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-120"}]: dispatch 2024-03-21T13:05:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:40 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-120", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:05:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:40 smithi155 ceph-mon[33846]: osdmap e606: 8 total, 8 up, 8 in 2024-03-21T13:05:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:40 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-120"}]: dispatch 2024-03-21T13:05:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:40 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-120"}]: dispatch 2024-03-21T13:05:41.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:41 smithi043 ceph-mon[30062]: pgmap v939: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:05:41.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:41 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-120"}]': finished 2024-03-21T13:05:41.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:41 smithi043 ceph-mon[30062]: osdmap e607: 8 total, 8 up, 8 in 2024-03-21T13:05:41.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-120", "mode": "writeback"}]: dispatch 2024-03-21T13:05:41.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:41 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-120", "mode": "writeback"}]: dispatch 2024-03-21T13:05:41.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:41 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:05:41.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:41 smithi043 ceph-mon[41699]: pgmap v939: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:05:41.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:41 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-120"}]': finished 2024-03-21T13:05:41.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:41 smithi043 ceph-mon[41699]: osdmap e607: 8 total, 8 up, 8 in 2024-03-21T13:05:41.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-120", "mode": "writeback"}]: dispatch 2024-03-21T13:05:41.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:41 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-120", "mode": "writeback"}]: dispatch 2024-03-21T13:05:41.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:41 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:05:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:41 smithi155 ceph-mon[33846]: pgmap v939: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:05:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:41 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-120"}]': finished 2024-03-21T13:05:42.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:41 smithi155 ceph-mon[33846]: osdmap e607: 8 total, 8 up, 8 in 2024-03-21T13:05:42.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-120", "mode": "writeback"}]: dispatch 2024-03-21T13:05:42.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:41 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-120", "mode": "writeback"}]: dispatch 2024-03-21T13:05:42.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:41 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:05:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:42 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-120", "mode": "writeback"}]': finished 2024-03-21T13:05:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:42 smithi043 ceph-mon[30062]: osdmap e608: 8 total, 8 up, 8 in 2024-03-21T13:05:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:42 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:42 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:05:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:42 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:05:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:42 smithi043 ceph-mon[30062]: osdmap e609: 8 total, 8 up, 8 in 2024-03-21T13:05:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-120"}]: dispatch 2024-03-21T13:05:42.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:42 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-120"}]: dispatch 2024-03-21T13:05:42.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:42 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-120", "mode": "writeback"}]': finished 2024-03-21T13:05:42.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:42 smithi043 ceph-mon[41699]: osdmap e608: 8 total, 8 up, 8 in 2024-03-21T13:05:42.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:42.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:42 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:42.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:42 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:05:42.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:42 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:05:42.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:42 smithi043 ceph-mon[41699]: osdmap e609: 8 total, 8 up, 8 in 2024-03-21T13:05:42.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-120"}]: dispatch 2024-03-21T13:05:42.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:42 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-120"}]: dispatch 2024-03-21T13:05:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:42 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-120", "mode": "writeback"}]': finished 2024-03-21T13:05:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:42 smithi155 ceph-mon[33846]: osdmap e608: 8 total, 8 up, 8 in 2024-03-21T13:05:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:42 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:42 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:05:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:42 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:05:43.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:42 smithi155 ceph-mon[33846]: osdmap e609: 8 total, 8 up, 8 in 2024-03-21T13:05:43.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-120"}]: dispatch 2024-03-21T13:05:43.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:42 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-120"}]: dispatch 2024-03-21T13:05:43.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:43 smithi043 ceph-mon[30062]: pgmap v942: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:05:43.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:43 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:05:43.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:43 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-120"}]': finished 2024-03-21T13:05:43.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:43 smithi043 ceph-mon[30062]: osdmap e610: 8 total, 8 up, 8 in 2024-03-21T13:05:43.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:43 smithi043 ceph-mon[41699]: pgmap v942: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:05:43.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:43 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:05:43.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:43 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-120"}]': finished 2024-03-21T13:05:43.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:43 smithi043 ceph-mon[41699]: osdmap e610: 8 total, 8 up, 8 in 2024-03-21T13:05:44.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:43 smithi155 ceph-mon[33846]: pgmap v942: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:05:44.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:43 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:05:44.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:43 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-120"}]': finished 2024-03-21T13:05:44.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:43 smithi155 ceph-mon[33846]: osdmap e610: 8 total, 8 up, 8 in 2024-03-21T13:05:45.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:45 smithi155 ceph-mon[33846]: pgmap v945: 332 pgs: 332 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-21T13:05:45.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:45 smithi155 ceph-mon[33846]: osdmap e611: 8 total, 8 up, 8 in 2024-03-21T13:05:45.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:45 smithi043 ceph-mon[30062]: pgmap v945: 332 pgs: 332 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-21T13:05:45.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:45 smithi043 ceph-mon[30062]: osdmap e611: 8 total, 8 up, 8 in 2024-03-21T13:05:45.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:45 smithi043 ceph-mon[41699]: pgmap v945: 332 pgs: 332 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-21T13:05:45.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:45 smithi043 ceph-mon[41699]: osdmap e611: 8 total, 8 up, 8 in 2024-03-21T13:05:46.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:46 smithi155 ceph-mon[33846]: osdmap e612: 8 total, 8 up, 8 in 2024-03-21T13:05:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:46 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:46.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:46 smithi043 ceph-mon[30062]: osdmap e612: 8 total, 8 up, 8 in 2024-03-21T13:05:46.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:46.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:46 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:46.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:46 smithi043 ceph-mon[41699]: osdmap e612: 8 total, 8 up, 8 in 2024-03-21T13:05:46.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:46.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:46 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:47.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:47 smithi155 ceph-mon[33846]: pgmap v948: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:05:47.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:47 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:05:47.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:47 smithi155 ceph-mon[33846]: osdmap e613: 8 total, 8 up, 8 in 2024-03-21T13:05:47.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:47.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:47 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:47.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:47 smithi155 ceph-mon[33846]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:05:47.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:47 smithi043 ceph-mon[30062]: pgmap v948: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:05:47.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:47 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:05:47.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:47 smithi043 ceph-mon[30062]: osdmap e613: 8 total, 8 up, 8 in 2024-03-21T13:05:47.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:47.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:47 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:47.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:47 smithi043 ceph-mon[30062]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:05:47.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:47 smithi043 ceph-mon[41699]: pgmap v948: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:05:47.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:47 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:05:47.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:47 smithi043 ceph-mon[41699]: osdmap e613: 8 total, 8 up, 8 in 2024-03-21T13:05:47.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:47.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:47 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:47.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:47 smithi043 ceph-mon[41699]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:05:48.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:05:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:05:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:05:48.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:05:48 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:05:48.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:48 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-122", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:05:48.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:48 smithi155 ceph-mon[33846]: osdmap e614: 8 total, 8 up, 8 in 2024-03-21T13:05:48.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:48 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-122"}]: dispatch 2024-03-21T13:05:48.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:48 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-122"}]: dispatch 2024-03-21T13:05:48.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:48 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-122", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:05:48.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:48 smithi043 ceph-mon[30062]: osdmap e614: 8 total, 8 up, 8 in 2024-03-21T13:05:48.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:48 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-122"}]: dispatch 2024-03-21T13:05:48.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:48 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-122"}]: dispatch 2024-03-21T13:05:48.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:48 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-122", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:05:48.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:48 smithi043 ceph-mon[41699]: osdmap e614: 8 total, 8 up, 8 in 2024-03-21T13:05:48.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:48 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-122"}]: dispatch 2024-03-21T13:05:48.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:48 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-122"}]: dispatch 2024-03-21T13:05:49.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:49 smithi155 ceph-mon[33846]: pgmap v951: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:05:49.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:49 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-122"}]': finished 2024-03-21T13:05:49.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:49 smithi155 ceph-mon[33846]: osdmap e615: 8 total, 8 up, 8 in 2024-03-21T13:05:49.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:49 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-122", "mode": "writeback"}]: dispatch 2024-03-21T13:05:49.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:49 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-122", "mode": "writeback"}]: dispatch 2024-03-21T13:05:49.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:49 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:05:49.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:49 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:05:49.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:49 smithi043 ceph-mon[30062]: pgmap v951: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:05:49.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:49 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-122"}]': finished 2024-03-21T13:05:49.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:49 smithi043 ceph-mon[30062]: osdmap e615: 8 total, 8 up, 8 in 2024-03-21T13:05:49.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:49 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-122", "mode": "writeback"}]: dispatch 2024-03-21T13:05:49.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:49 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-122", "mode": "writeback"}]: dispatch 2024-03-21T13:05:49.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:49 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:05:49.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:49 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:05:49.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:49 smithi043 ceph-mon[41699]: pgmap v951: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 632 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:05:49.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:49 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-122"}]': finished 2024-03-21T13:05:49.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:49 smithi043 ceph-mon[41699]: osdmap e615: 8 total, 8 up, 8 in 2024-03-21T13:05:49.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:49 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-122", "mode": "writeback"}]: dispatch 2024-03-21T13:05:49.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:49 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-122", "mode": "writeback"}]: dispatch 2024-03-21T13:05:49.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:49 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:05:49.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:49 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:05:50.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:50 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:05:50.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:50 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:05:50.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:50 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-122", "mode": "writeback"}]': finished 2024-03-21T13:05:50.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:50 smithi155 ceph-mon[33846]: osdmap e616: 8 total, 8 up, 8 in 2024-03-21T13:05:50.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:50 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:50.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:50 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:50.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:50 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:05:50.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:50 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:05:50.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:50 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-122", "mode": "writeback"}]': finished 2024-03-21T13:05:50.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:50 smithi043 ceph-mon[30062]: osdmap e616: 8 total, 8 up, 8 in 2024-03-21T13:05:50.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:50 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:50.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:50 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:50.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:50 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:05:50.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:50 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:05:50.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:50 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-122", "mode": "writeback"}]': finished 2024-03-21T13:05:50.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:50 smithi043 ceph-mon[41699]: osdmap e616: 8 total, 8 up, 8 in 2024-03-21T13:05:50.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:50 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:50.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:50 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:05:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:51 smithi043 ceph-mon[30062]: pgmap v954: 332 pgs: 332 active+clean; 582 KiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:05:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:51 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:05:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:51 smithi043 ceph-mon[30062]: osdmap e617: 8 total, 8 up, 8 in 2024-03-21T13:05:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:51 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-122"}]: dispatch 2024-03-21T13:05:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:51 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-122"}]: dispatch 2024-03-21T13:05:51.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:51 smithi043 ceph-mon[41699]: pgmap v954: 332 pgs: 332 active+clean; 582 KiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:05:51.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:51 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:05:51.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:51 smithi043 ceph-mon[41699]: osdmap e617: 8 total, 8 up, 8 in 2024-03-21T13:05:51.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:51 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-122"}]: dispatch 2024-03-21T13:05:51.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:51 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-122"}]: dispatch 2024-03-21T13:05:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:51 smithi155 ceph-mon[33846]: pgmap v954: 332 pgs: 332 active+clean; 582 KiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:05:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:51 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:05:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:51 smithi155 ceph-mon[33846]: osdmap e617: 8 total, 8 up, 8 in 2024-03-21T13:05:52.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:51 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-122"}]: dispatch 2024-03-21T13:05:52.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:51 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-122"}]: dispatch 2024-03-21T13:05:52.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:52 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:05:52.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:52 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-122"}]': finished 2024-03-21T13:05:52.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:52 smithi043 ceph-mon[30062]: osdmap e618: 8 total, 8 up, 8 in 2024-03-21T13:05:52.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:52 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:05:52.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:52 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:05:52.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:52 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-122"}]': finished 2024-03-21T13:05:52.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:52 smithi043 ceph-mon[41699]: osdmap e618: 8 total, 8 up, 8 in 2024-03-21T13:05:52.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:52 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:05:53.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:52 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:05:53.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:52 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-122"}]': finished 2024-03-21T13:05:53.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:52 smithi155 ceph-mon[33846]: osdmap e618: 8 total, 8 up, 8 in 2024-03-21T13:05:53.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:52 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:05:53.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:53 smithi043 ceph-mon[30062]: pgmap v957: 332 pgs: 332 active+clean; 582 KiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:05:53.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:53 smithi043 ceph-mon[30062]: osdmap e619: 8 total, 8 up, 8 in 2024-03-21T13:05:53.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:53 smithi043 ceph-mon[41699]: pgmap v957: 332 pgs: 332 active+clean; 582 KiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:05:53.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:53 smithi043 ceph-mon[41699]: osdmap e619: 8 total, 8 up, 8 in 2024-03-21T13:05:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:53 smithi155 ceph-mon[33846]: pgmap v957: 332 pgs: 332 active+clean; 582 KiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:05:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:53 smithi155 ceph-mon[33846]: osdmap e619: 8 total, 8 up, 8 in 2024-03-21T13:05:54.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:54 smithi043 ceph-mon[30062]: osdmap e620: 8 total, 8 up, 8 in 2024-03-21T13:05:54.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:54 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:54.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:54 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:54.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:54 smithi043 ceph-mon[41699]: osdmap e620: 8 total, 8 up, 8 in 2024-03-21T13:05:54.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:54 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:54.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:54 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:55.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:54 smithi155 ceph-mon[33846]: osdmap e620: 8 total, 8 up, 8 in 2024-03-21T13:05:55.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:54 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:55.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:54 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:05:55.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:55 smithi043 ceph-mon[30062]: pgmap v960: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:05:55.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:55 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:05:55.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:55 smithi043 ceph-mon[30062]: osdmap e621: 8 total, 8 up, 8 in 2024-03-21T13:05:55.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:55 smithi043 ceph-mon[41699]: pgmap v960: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:05:55.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:55 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:05:55.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:55 smithi043 ceph-mon[41699]: osdmap e621: 8 total, 8 up, 8 in 2024-03-21T13:05:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:55 smithi155 ceph-mon[33846]: pgmap v960: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:05:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:55 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:05:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:55 smithi155 ceph-mon[33846]: osdmap e621: 8 total, 8 up, 8 in 2024-03-21T13:05:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:56 smithi155 ceph-mon[33846]: osdmap e622: 8 total, 8 up, 8 in 2024-03-21T13:05:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:56 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:56 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:57.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:56 smithi043 ceph-mon[30062]: osdmap e622: 8 total, 8 up, 8 in 2024-03-21T13:05:57.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:56 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:57.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:56 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:56 smithi043 ceph-mon[41699]: osdmap e622: 8 total, 8 up, 8 in 2024-03-21T13:05:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:56 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:57.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:56 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:05:58.023 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:57 smithi043 ceph-mon[30062]: pgmap v963: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:05:58.023 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:57 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-124", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:05:58.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:57 smithi043 ceph-mon[30062]: osdmap e623: 8 total, 8 up, 8 in 2024-03-21T13:05:58.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:57 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-124"}]: dispatch 2024-03-21T13:05:58.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:57 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-124"}]: dispatch 2024-03-21T13:05:58.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:57 smithi043 ceph-mon[41699]: pgmap v963: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:05:58.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:57 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-124", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:05:58.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:57 smithi043 ceph-mon[41699]: osdmap e623: 8 total, 8 up, 8 in 2024-03-21T13:05:58.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:57 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-124"}]: dispatch 2024-03-21T13:05:58.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:57 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-124"}]: dispatch 2024-03-21T13:05:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:57 smithi155 ceph-mon[33846]: pgmap v963: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:05:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:57 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-124", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:05:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:57 smithi155 ceph-mon[33846]: osdmap e623: 8 total, 8 up, 8 in 2024-03-21T13:05:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:57 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-124"}]: dispatch 2024-03-21T13:05:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:57 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-124"}]: dispatch 2024-03-21T13:05:58.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:05:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:05:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:05:59.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:05:58 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:05:59.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:58 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-124"}]': finished 2024-03-21T13:05:59.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:58 smithi155 ceph-mon[33846]: osdmap e624: 8 total, 8 up, 8 in 2024-03-21T13:05:59.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:58 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-124", "mode": "writeback"}]: dispatch 2024-03-21T13:05:59.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:58 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-124", "mode": "writeback"}]: dispatch 2024-03-21T13:05:59.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:58 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-124"}]': finished 2024-03-21T13:05:59.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:58 smithi043 ceph-mon[30062]: osdmap e624: 8 total, 8 up, 8 in 2024-03-21T13:05:59.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:58 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-124", "mode": "writeback"}]: dispatch 2024-03-21T13:05:59.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:58 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-124", "mode": "writeback"}]: dispatch 2024-03-21T13:05:59.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:58 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-124"}]': finished 2024-03-21T13:05:59.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:58 smithi043 ceph-mon[41699]: osdmap e624: 8 total, 8 up, 8 in 2024-03-21T13:05:59.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:58 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-124", "mode": "writeback"}]: dispatch 2024-03-21T13:05:59.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:58 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-124", "mode": "writeback"}]: dispatch 2024-03-21T13:06:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:59 smithi155 ceph-mon[33846]: pgmap v966: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 651 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:06:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:59 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:06:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:59 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-124", "mode": "writeback"}]': finished 2024-03-21T13:06:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:05:59 smithi155 ceph-mon[33846]: osdmap e625: 8 total, 8 up, 8 in 2024-03-21T13:06:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:59 smithi043 ceph-mon[30062]: pgmap v966: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 651 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:06:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:59 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:06:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:59 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-124", "mode": "writeback"}]': finished 2024-03-21T13:06:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:05:59 smithi043 ceph-mon[30062]: osdmap e625: 8 total, 8 up, 8 in 2024-03-21T13:06:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:59 smithi043 ceph-mon[41699]: pgmap v966: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 651 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:06:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:59 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:06:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:59 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-124", "mode": "writeback"}]': finished 2024-03-21T13:06:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:05:59 smithi043 ceph-mon[41699]: osdmap e625: 8 total, 8 up, 8 in 2024-03-21T13:06:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:00 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:06:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:00 smithi155 ceph-mon[33846]: osdmap e626: 8 total, 8 up, 8 in 2024-03-21T13:06:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:00 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:00 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:00 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:06:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:00 smithi043 ceph-mon[30062]: osdmap e626: 8 total, 8 up, 8 in 2024-03-21T13:06:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:00 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:00 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:00 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:06:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:00 smithi043 ceph-mon[41699]: osdmap e626: 8 total, 8 up, 8 in 2024-03-21T13:06:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:00 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:00 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:01 smithi155 ceph-mon[33846]: pgmap v969: 332 pgs: 332 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T13:06:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:01 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:06:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:01 smithi155 ceph-mon[33846]: osdmap e627: 8 total, 8 up, 8 in 2024-03-21T13:06:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:01 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-124"}]: dispatch 2024-03-21T13:06:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:01 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-124"}]: dispatch 2024-03-21T13:06:02.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:01 smithi043 ceph-mon[30062]: pgmap v969: 332 pgs: 332 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T13:06:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:01 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:06:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:01 smithi043 ceph-mon[30062]: osdmap e627: 8 total, 8 up, 8 in 2024-03-21T13:06:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:01 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-124"}]: dispatch 2024-03-21T13:06:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:01 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-124"}]: dispatch 2024-03-21T13:06:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:01 smithi043 ceph-mon[41699]: pgmap v969: 332 pgs: 332 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T13:06:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:01 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:06:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:01 smithi043 ceph-mon[41699]: osdmap e627: 8 total, 8 up, 8 in 2024-03-21T13:06:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:01 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-124"}]: dispatch 2024-03-21T13:06:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:01 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-124"}]: dispatch 2024-03-21T13:06:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:02 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:06:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:02 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-124"}]': finished 2024-03-21T13:06:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:02 smithi155 ceph-mon[33846]: osdmap e628: 8 total, 8 up, 8 in 2024-03-21T13:06:03.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:02 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:06:03.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:02 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-124"}]': finished 2024-03-21T13:06:03.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:02 smithi043 ceph-mon[30062]: osdmap e628: 8 total, 8 up, 8 in 2024-03-21T13:06:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:02 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:06:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:02 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-124"}]': finished 2024-03-21T13:06:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:02 smithi043 ceph-mon[41699]: osdmap e628: 8 total, 8 up, 8 in 2024-03-21T13:06:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:03 smithi155 ceph-mon[33846]: pgmap v972: 332 pgs: 332 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T13:06:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:03 smithi155 ceph-mon[33846]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:06:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:03 smithi155 ceph-mon[33846]: osdmap e629: 8 total, 8 up, 8 in 2024-03-21T13:06:04.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:03 smithi043 ceph-mon[30062]: pgmap v972: 332 pgs: 332 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T13:06:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:03 smithi043 ceph-mon[30062]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:06:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:03 smithi043 ceph-mon[30062]: osdmap e629: 8 total, 8 up, 8 in 2024-03-21T13:06:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:03 smithi043 ceph-mon[41699]: pgmap v972: 332 pgs: 332 active+clean; 583 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-21T13:06:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:03 smithi043 ceph-mon[41699]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:06:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:03 smithi043 ceph-mon[41699]: osdmap e629: 8 total, 8 up, 8 in 2024-03-21T13:06:05.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:04 smithi155 ceph-mon[33846]: osdmap e630: 8 total, 8 up, 8 in 2024-03-21T13:06:05.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:05.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:04 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:05.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:04 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:06:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:04 smithi043 ceph-mon[30062]: osdmap e630: 8 total, 8 up, 8 in 2024-03-21T13:06:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:04 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:04 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:06:05.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:04 smithi043 ceph-mon[41699]: osdmap e630: 8 total, 8 up, 8 in 2024-03-21T13:06:05.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:05.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:04 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:05.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:04 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:06:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:05 smithi155 ceph-mon[33846]: pgmap v975: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:06:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:05 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:06:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:05 smithi155 ceph-mon[33846]: osdmap e631: 8 total, 8 up, 8 in 2024-03-21T13:06:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:05 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:06.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:05 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:05 smithi043 ceph-mon[30062]: pgmap v975: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:06:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:05 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:06:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:05 smithi043 ceph-mon[30062]: osdmap e631: 8 total, 8 up, 8 in 2024-03-21T13:06:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:05 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:05 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:05 smithi043 ceph-mon[41699]: pgmap v975: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:06:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:05 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:06:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:05 smithi043 ceph-mon[41699]: osdmap e631: 8 total, 8 up, 8 in 2024-03-21T13:06:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:05 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:06.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:05 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:07.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:06 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-126", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:06:07.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:06 smithi155 ceph-mon[33846]: osdmap e632: 8 total, 8 up, 8 in 2024-03-21T13:06:07.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:06 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-126"}]: dispatch 2024-03-21T13:06:07.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:06 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-126"}]: dispatch 2024-03-21T13:06:07.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:06 smithi155 ceph-mon[33846]: pgmap v978: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:06:07.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:06 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:06:07.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:06 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-126"}]': finished 2024-03-21T13:06:07.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:06 smithi155 ceph-mon[33846]: osdmap e633: 8 total, 8 up, 8 in 2024-03-21T13:06:07.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:06 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-126", "mode": "writeback"}]: dispatch 2024-03-21T13:06:07.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:06 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-126", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:06:07.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:06 smithi043 ceph-mon[30062]: osdmap e632: 8 total, 8 up, 8 in 2024-03-21T13:06:07.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:06 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-126"}]: dispatch 2024-03-21T13:06:07.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:06 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-126"}]: dispatch 2024-03-21T13:06:07.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:06 smithi043 ceph-mon[30062]: pgmap v978: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:06:07.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:06 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:06:07.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:06 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-126"}]': finished 2024-03-21T13:06:07.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:06 smithi043 ceph-mon[30062]: osdmap e633: 8 total, 8 up, 8 in 2024-03-21T13:06:07.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:06 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-126", "mode": "writeback"}]: dispatch 2024-03-21T13:06:07.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:06 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-126", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:06:07.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:06 smithi043 ceph-mon[41699]: osdmap e632: 8 total, 8 up, 8 in 2024-03-21T13:06:07.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:06 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-126"}]: dispatch 2024-03-21T13:06:07.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:06 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-126"}]: dispatch 2024-03-21T13:06:07.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:06 smithi043 ceph-mon[41699]: pgmap v978: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:06:07.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:06 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:06:07.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:06 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-126"}]': finished 2024-03-21T13:06:07.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:06 smithi043 ceph-mon[41699]: osdmap e633: 8 total, 8 up, 8 in 2024-03-21T13:06:07.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:06 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-126", "mode": "writeback"}]: dispatch 2024-03-21T13:06:08.134 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:06:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:06:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:06:08.134 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:07 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-126", "mode": "writeback"}]: dispatch 2024-03-21T13:06:08.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:07 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:06:08.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:07 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:06:08.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:07 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:06:08.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:07 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:06:08.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:07 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-126", "mode": "writeback"}]': finished 2024-03-21T13:06:08.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:07 smithi043 ceph-mon[30062]: osdmap e634: 8 total, 8 up, 8 in 2024-03-21T13:06:08.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:07 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:08.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:07 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:08.135 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:07 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-126", "mode": "writeback"}]: dispatch 2024-03-21T13:06:08.135 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:07 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:06:08.135 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:07 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:06:08.135 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:07 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:06:08.136 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:07 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:06:08.136 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:07 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-126", "mode": "writeback"}]': finished 2024-03-21T13:06:08.136 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:07 smithi043 ceph-mon[41699]: osdmap e634: 8 total, 8 up, 8 in 2024-03-21T13:06:08.136 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:07 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:08.136 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:07 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:07 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-126", "mode": "writeback"}]: dispatch 2024-03-21T13:06:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:07 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:06:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:07 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:06:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:07 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:06:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:07 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:06:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:07 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-126", "mode": "writeback"}]': finished 2024-03-21T13:06:08.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:07 smithi155 ceph-mon[33846]: osdmap e634: 8 total, 8 up, 8 in 2024-03-21T13:06:08.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:07 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:08.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:07 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:09.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:06:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:06:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:09 smithi155 ceph-mon[33846]: pgmap v981: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:06:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:09 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:06:09.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:09 smithi155 ceph-mon[33846]: osdmap e635: 8 total, 8 up, 8 in 2024-03-21T13:06:09.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:09 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-126"}]: dispatch 2024-03-21T13:06:09.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:09 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-126"}]: dispatch 2024-03-21T13:06:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:09 smithi043 ceph-mon[30062]: pgmap v981: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:06:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:09 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:06:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:09 smithi043 ceph-mon[30062]: osdmap e635: 8 total, 8 up, 8 in 2024-03-21T13:06:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:09 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-126"}]: dispatch 2024-03-21T13:06:09.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:09 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-126"}]: dispatch 2024-03-21T13:06:09.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:09 smithi043 ceph-mon[41699]: pgmap v981: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 656 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:06:09.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:09 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:06:09.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:09 smithi043 ceph-mon[41699]: osdmap e635: 8 total, 8 up, 8 in 2024-03-21T13:06:09.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:09 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-126"}]: dispatch 2024-03-21T13:06:09.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:09 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-126"}]: dispatch 2024-03-21T13:06:10.686 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 seapi_tier_pp: require_osd_release = squid 2024-03-21T13:06:10.686 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9055 ms) 2024-03-21T13:06:10.686 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-03-21T13:06:10.686 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:06:10.686 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6034 ms) 2024-03-21T13:06:10.686 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-03-21T13:06:10.686 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:06:10.687 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (5040 ms) 2024-03-21T13:06:10.687 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-03-21T13:06:10.687 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:06:10.687 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (24700 ms) 2024-03-21T13:06:10.687 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-03-21T13:06:10.687 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:06:10.687 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (14057 ms) 2024-03-21T13:06:10.687 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-03-21T13:06:10.687 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12050 ms) 2024-03-21T13:06:10.687 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-03-21T13:06:10.687 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-03-21T13:06:10.687 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 00000010 69 73 |is| 2024-03-21T13:06:10.687 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 00000012 2024-03-21T13:06:10.687 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12061 ms) 2024-03-21T13:06:10.687 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-03-21T13:06:10.688 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: require_osd_release = squid 2024-03-21T13:06:10.688 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6040 ms) 2024-03-21T13:06:10.688 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (551459 ms total) 2024-03-21T13:06:10.688 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 2024-03-21T13:06:10.688 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-03-21T13:06:10.688 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-03-21T13:06:10.688 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1095 ms) 2024-03-21T13:06:10.688 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-03-21T13:06:10.688 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (10982 ms) 2024-03-21T13:06:10.688 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-03-21T13:06:10.688 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18077 ms) 2024-03-21T13:06:10.688 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-03-21T13:06:10.688 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (2 ms) 2024-03-21T13:06:10.688 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30156 ms total) 2024-03-21T13:06:10.688 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 2024-03-21T13:06:10.689 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-03-21T13:06:10.689 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-03-21T13:06:10.689 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7027 ms) 2024-03-21T13:06:10.689 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-03-21T13:06:10.689 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8057 ms) 2024-03-21T13:06:10.689 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-03-21T13:06:10.689 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: waiting for scrub... 2024-03-21T13:06:10.689 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: done waiting 2024-03-21T13:06:10.689 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24477 ms) 2024-03-21T13:06:10.689 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-03-21T13:06:10.689 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (10079 ms) 2024-03-21T13:06:10.689 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-03-21T13:06:10.689 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (7969 ms) 2024-03-21T13:06:10.689 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-03-21T13:06:10.689 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8054 ms) 2024-03-21T13:06:10.690 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-03-21T13:06:10.690 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (10067 ms) 2024-03-21T13:06:10.690 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-03-21T13:06:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:10 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:06:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:10 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:06:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:10 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-126"}]': finished 2024-03-21T13:06:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:10 smithi155 ceph-mon[33846]: osdmap e636: 8 total, 8 up, 8 in 2024-03-21T13:06:10.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:10 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:06:10.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:10 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:06:10.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:10 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-126"}]': finished 2024-03-21T13:06:10.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:10 smithi043 ceph-mon[30062]: osdmap e636: 8 total, 8 up, 8 in 2024-03-21T13:06:10.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:10 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:06:10.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:10 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:06:10.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:10 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-126"}]': finished 2024-03-21T13:06:10.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:10 smithi043 ceph-mon[41699]: osdmap e636: 8 total, 8 up, 8 in 2024-03-21T13:06:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:11 smithi155 ceph-mon[33846]: pgmap v984: 332 pgs: 332 active+clean; 582 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:06:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:11 smithi155 ceph-mon[33846]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:06:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:11 smithi155 ceph-mon[33846]: osdmap e637: 8 total, 8 up, 8 in 2024-03-21T13:06:11.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:11 smithi043 ceph-mon[30062]: pgmap v984: 332 pgs: 332 active+clean; 582 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:06:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:11 smithi043 ceph-mon[30062]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:06:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:11 smithi043 ceph-mon[30062]: osdmap e637: 8 total, 8 up, 8 in 2024-03-21T13:06:11.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:11 smithi043 ceph-mon[41699]: pgmap v984: 332 pgs: 332 active+clean; 582 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-21T13:06:11.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:11 smithi043 ceph-mon[41699]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:06:11.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:11 smithi043 ceph-mon[41699]: osdmap e637: 8 total, 8 up, 8 in 2024-03-21T13:06:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:12 smithi155 ceph-mon[33846]: osdmap e638: 8 total, 8 up, 8 in 2024-03-21T13:06:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:12 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:12.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:12 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:12 smithi043 ceph-mon[30062]: osdmap e638: 8 total, 8 up, 8 in 2024-03-21T13:06:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:12 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:12.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:12 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:12.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:12 smithi043 ceph-mon[41699]: osdmap e638: 8 total, 8 up, 8 in 2024-03-21T13:06:12.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:12 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:12.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:12 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:13 smithi155 ceph-mon[33846]: pgmap v987: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:06:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:13 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:06:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:13 smithi155 ceph-mon[33846]: osdmap e639: 8 total, 8 up, 8 in 2024-03-21T13:06:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:13 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:13 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:13.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:13 smithi043 ceph-mon[30062]: pgmap v987: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:06:13.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:13 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:06:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:13 smithi043 ceph-mon[30062]: osdmap e639: 8 total, 8 up, 8 in 2024-03-21T13:06:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:13 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:13 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:13.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:13 smithi043 ceph-mon[41699]: pgmap v987: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 657 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:06:13.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:13 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:06:13.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:13 smithi043 ceph-mon[41699]: osdmap e639: 8 total, 8 up, 8 in 2024-03-21T13:06:13.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:13 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:13.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:13 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:14.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:14 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-128", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:06:14.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:14 smithi155 ceph-mon[33846]: osdmap e640: 8 total, 8 up, 8 in 2024-03-21T13:06:14.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:14 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-128"}]: dispatch 2024-03-21T13:06:14.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:14 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-128"}]: dispatch 2024-03-21T13:06:14.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:14 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-128"}]': finished 2024-03-21T13:06:14.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:14 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-128", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:06:14.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:14 smithi043 ceph-mon[30062]: osdmap e640: 8 total, 8 up, 8 in 2024-03-21T13:06:14.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:14 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-128"}]: dispatch 2024-03-21T13:06:14.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:14 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-128"}]: dispatch 2024-03-21T13:06:14.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:14 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-128"}]': finished 2024-03-21T13:06:14.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:14 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-128", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:06:14.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:14 smithi043 ceph-mon[41699]: osdmap e640: 8 total, 8 up, 8 in 2024-03-21T13:06:14.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:14 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-128"}]: dispatch 2024-03-21T13:06:14.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:14 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-128"}]: dispatch 2024-03-21T13:06:14.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:14 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-128"}]': finished 2024-03-21T13:06:15.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:15 smithi155 ceph-mon[33846]: pgmap v990: 332 pgs: 332 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-21T13:06:15.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:15 smithi155 ceph-mon[33846]: osdmap e641: 8 total, 8 up, 8 in 2024-03-21T13:06:15.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:15 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-128", "mode": "writeback"}]: dispatch 2024-03-21T13:06:15.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:15 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-128", "mode": "writeback"}]: dispatch 2024-03-21T13:06:15.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:15 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:06:15.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:15 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-128", "mode": "writeback"}]': finished 2024-03-21T13:06:15.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:15 smithi155 ceph-mon[33846]: osdmap e642: 8 total, 8 up, 8 in 2024-03-21T13:06:15.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:15 smithi043 ceph-mon[30062]: pgmap v990: 332 pgs: 332 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-21T13:06:15.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:15 smithi043 ceph-mon[30062]: osdmap e641: 8 total, 8 up, 8 in 2024-03-21T13:06:15.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:15 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-128", "mode": "writeback"}]: dispatch 2024-03-21T13:06:15.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:15 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-128", "mode": "writeback"}]: dispatch 2024-03-21T13:06:15.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:15 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:06:15.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:15 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-128", "mode": "writeback"}]': finished 2024-03-21T13:06:15.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:15 smithi043 ceph-mon[30062]: osdmap e642: 8 total, 8 up, 8 in 2024-03-21T13:06:15.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:15 smithi043 ceph-mon[41699]: pgmap v990: 332 pgs: 332 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-21T13:06:15.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:15 smithi043 ceph-mon[41699]: osdmap e641: 8 total, 8 up, 8 in 2024-03-21T13:06:15.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:15 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-128", "mode": "writeback"}]: dispatch 2024-03-21T13:06:15.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:15 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-128", "mode": "writeback"}]: dispatch 2024-03-21T13:06:15.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:15 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:06:15.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:15 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-128", "mode": "writeback"}]': finished 2024-03-21T13:06:15.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:15 smithi043 ceph-mon[41699]: osdmap e642: 8 total, 8 up, 8 in 2024-03-21T13:06:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:17 smithi043 ceph-mon[30062]: pgmap v993: 332 pgs: 332 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:06:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:17 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:06:17.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:17 smithi043 ceph-mon[41699]: pgmap v993: 332 pgs: 332 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:06:17.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:17 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:06:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:17 smithi155 ceph-mon[33846]: pgmap v993: 332 pgs: 332 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:06:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:17 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:06:18.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:06:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:06:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:06:19.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:06:18 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:06:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:19 smithi043 ceph-mon[30062]: pgmap v994: 332 pgs: 332 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 939 B/s rd, 0 op/s 2024-03-21T13:06:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:19 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:06:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:19 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:06:19.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:19 smithi043 ceph-mon[41699]: pgmap v994: 332 pgs: 332 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 939 B/s rd, 0 op/s 2024-03-21T13:06:19.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:19 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:06:19.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:19 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:06:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:19 smithi155 ceph-mon[33846]: pgmap v994: 332 pgs: 332 active+clean; 582 KiB data, 679 MiB used, 715 GiB / 715 GiB avail; 939 B/s rd, 0 op/s 2024-03-21T13:06:20.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:19 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:06:20.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:19 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:06:20.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:20 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:06:20.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:20 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:06:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:20 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:06:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:21 smithi043 ceph-mon[30062]: pgmap v995: 332 pgs: 332 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 952 B/s rd, 317 B/s wr, 1 op/s 2024-03-21T13:06:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:21 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:21 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:21.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:21 smithi043 ceph-mon[41699]: pgmap v995: 332 pgs: 332 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 952 B/s rd, 317 B/s wr, 1 op/s 2024-03-21T13:06:21.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:21 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:21.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:21 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:22.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:21 smithi155 ceph-mon[33846]: pgmap v995: 332 pgs: 332 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 952 B/s rd, 317 B/s wr, 1 op/s 2024-03-21T13:06:22.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:21 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:22.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:21 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:22 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:06:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:22 smithi043 ceph-mon[30062]: osdmap e643: 8 total, 8 up, 8 in 2024-03-21T13:06:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:22 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-128"}]: dispatch 2024-03-21T13:06:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:22 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-128"}]: dispatch 2024-03-21T13:06:22.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:22 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:06:22.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:22 smithi043 ceph-mon[41699]: osdmap e643: 8 total, 8 up, 8 in 2024-03-21T13:06:22.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:22 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-128"}]: dispatch 2024-03-21T13:06:22.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:22 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-128"}]: dispatch 2024-03-21T13:06:23.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:22 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:06:23.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:22 smithi155 ceph-mon[33846]: osdmap e643: 8 total, 8 up, 8 in 2024-03-21T13:06:23.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:22 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-128"}]: dispatch 2024-03-21T13:06:23.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:22 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-128"}]: dispatch 2024-03-21T13:06:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:23 smithi155 ceph-mon[33846]: pgmap v997: 332 pgs: 332 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 824 B/s rd, 274 B/s wr, 1 op/s 2024-03-21T13:06:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:23 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:06:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:23 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-128"}]': finished 2024-03-21T13:06:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:23 smithi155 ceph-mon[33846]: osdmap e644: 8 total, 8 up, 8 in 2024-03-21T13:06:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:23 smithi043 ceph-mon[30062]: pgmap v997: 332 pgs: 332 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 824 B/s rd, 274 B/s wr, 1 op/s 2024-03-21T13:06:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:23 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:06:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:23 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-128"}]': finished 2024-03-21T13:06:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:23 smithi043 ceph-mon[30062]: osdmap e644: 8 total, 8 up, 8 in 2024-03-21T13:06:24.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:23 smithi043 ceph-mon[41699]: pgmap v997: 332 pgs: 332 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 824 B/s rd, 274 B/s wr, 1 op/s 2024-03-21T13:06:24.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:23 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:06:24.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:23 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-128"}]': finished 2024-03-21T13:06:24.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:23 smithi043 ceph-mon[41699]: osdmap e644: 8 total, 8 up, 8 in 2024-03-21T13:06:25.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:24 smithi155 ceph-mon[33846]: osdmap e645: 8 total, 8 up, 8 in 2024-03-21T13:06:25.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:24 smithi043 ceph-mon[30062]: osdmap e645: 8 total, 8 up, 8 in 2024-03-21T13:06:25.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:24 smithi043 ceph-mon[41699]: osdmap e645: 8 total, 8 up, 8 in 2024-03-21T13:06:26.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:25 smithi155 ceph-mon[33846]: pgmap v1000: 300 pgs: 300 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:06:26.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:25 smithi155 ceph-mon[33846]: osdmap e646: 8 total, 8 up, 8 in 2024-03-21T13:06:26.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:25 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:26.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:25 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:26.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:25 smithi043 ceph-mon[30062]: pgmap v1000: 300 pgs: 300 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:06:26.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:25 smithi043 ceph-mon[30062]: osdmap e646: 8 total, 8 up, 8 in 2024-03-21T13:06:26.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:25 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:26.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:25 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:26.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:25 smithi043 ceph-mon[41699]: pgmap v1000: 300 pgs: 300 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:06:26.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:25 smithi043 ceph-mon[41699]: osdmap e646: 8 total, 8 up, 8 in 2024-03-21T13:06:26.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:25 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:26.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:25 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:27.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:26 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:06:27.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:26 smithi155 ceph-mon[33846]: osdmap e647: 8 total, 8 up, 8 in 2024-03-21T13:06:27.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:26 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:27.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:26 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:27.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:26 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:06:27.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:26 smithi043 ceph-mon[30062]: osdmap e647: 8 total, 8 up, 8 in 2024-03-21T13:06:27.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:26 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:27.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:26 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:27.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:26 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:06:27.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:26 smithi043 ceph-mon[41699]: osdmap e647: 8 total, 8 up, 8 in 2024-03-21T13:06:27.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:26 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:27.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:26 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:28.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:27 smithi043 ceph-mon[30062]: pgmap v1003: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:06:28.025 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:27 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-130", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:06:28.025 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:27 smithi043 ceph-mon[30062]: osdmap e648: 8 total, 8 up, 8 in 2024-03-21T13:06:28.025 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:27 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-130"}]: dispatch 2024-03-21T13:06:28.025 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:27 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-130"}]: dispatch 2024-03-21T13:06:28.025 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:27 smithi043 ceph-mon[41699]: pgmap v1003: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:06:28.025 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:27 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-130", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:06:28.025 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:27 smithi043 ceph-mon[41699]: osdmap e648: 8 total, 8 up, 8 in 2024-03-21T13:06:28.025 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:27 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-130"}]: dispatch 2024-03-21T13:06:28.025 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:27 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-130"}]: dispatch 2024-03-21T13:06:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:27 smithi155 ceph-mon[33846]: pgmap v1003: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:06:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:27 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-130", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:06:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:27 smithi155 ceph-mon[33846]: osdmap e648: 8 total, 8 up, 8 in 2024-03-21T13:06:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:27 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-130"}]: dispatch 2024-03-21T13:06:28.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:27 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-130"}]: dispatch 2024-03-21T13:06:28.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:06:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:06:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:06:29.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:06:28 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:06:29.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:28 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-130"}]': finished 2024-03-21T13:06:29.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:28 smithi155 ceph-mon[33846]: osdmap e649: 8 total, 8 up, 8 in 2024-03-21T13:06:29.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-130", "mode": "writeback"}]: dispatch 2024-03-21T13:06:29.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:28 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-130", "mode": "writeback"}]: dispatch 2024-03-21T13:06:29.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:28 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-130"}]': finished 2024-03-21T13:06:29.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:28 smithi043 ceph-mon[30062]: osdmap e649: 8 total, 8 up, 8 in 2024-03-21T13:06:29.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-130", "mode": "writeback"}]: dispatch 2024-03-21T13:06:29.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:28 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-130", "mode": "writeback"}]: dispatch 2024-03-21T13:06:29.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:28 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-130"}]': finished 2024-03-21T13:06:29.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:28 smithi043 ceph-mon[41699]: osdmap e649: 8 total, 8 up, 8 in 2024-03-21T13:06:29.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-130", "mode": "writeback"}]: dispatch 2024-03-21T13:06:29.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:28 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-130", "mode": "writeback"}]: dispatch 2024-03-21T13:06:30.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:29 smithi155 ceph-mon[33846]: pgmap v1006: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:06:30.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:29 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:06:30.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:29 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-130", "mode": "writeback"}]': finished 2024-03-21T13:06:30.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:29 smithi155 ceph-mon[33846]: osdmap e650: 8 total, 8 up, 8 in 2024-03-21T13:06:30.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:29 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:30.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:29 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:29 smithi043 ceph-mon[30062]: pgmap v1006: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:06:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:29 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:06:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:29 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-130", "mode": "writeback"}]': finished 2024-03-21T13:06:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:29 smithi043 ceph-mon[30062]: osdmap e650: 8 total, 8 up, 8 in 2024-03-21T13:06:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:29 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:29 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:30.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:29 smithi043 ceph-mon[41699]: pgmap v1006: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:06:30.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:29 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:06:30.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:29 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-130", "mode": "writeback"}]': finished 2024-03-21T13:06:30.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:29 smithi043 ceph-mon[41699]: osdmap e650: 8 total, 8 up, 8 in 2024-03-21T13:06:30.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:29 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:30.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:29 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:31.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:30 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:06:31.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:30 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:06:31.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:30 smithi155 ceph-mon[33846]: osdmap e651: 8 total, 8 up, 8 in 2024-03-21T13:06:31.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:30 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-130"}]: dispatch 2024-03-21T13:06:31.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:30 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-130"}]: dispatch 2024-03-21T13:06:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:30 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:06:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:30 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:06:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:30 smithi043 ceph-mon[30062]: osdmap e651: 8 total, 8 up, 8 in 2024-03-21T13:06:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:30 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-130"}]: dispatch 2024-03-21T13:06:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:30 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-130"}]: dispatch 2024-03-21T13:06:31.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:30 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:06:31.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:30 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:06:31.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:30 smithi043 ceph-mon[41699]: osdmap e651: 8 total, 8 up, 8 in 2024-03-21T13:06:31.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:30 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-130"}]: dispatch 2024-03-21T13:06:31.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:30 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-130"}]: dispatch 2024-03-21T13:06:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:31 smithi155 ceph-mon[33846]: pgmap v1009: 332 pgs: 332 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:06:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:31 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:06:32.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:31 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-130"}]': finished 2024-03-21T13:06:32.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:31 smithi155 ceph-mon[33846]: osdmap e652: 8 total, 8 up, 8 in 2024-03-21T13:06:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:31 smithi043 ceph-mon[30062]: pgmap v1009: 332 pgs: 332 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:06:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:31 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:06:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:31 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-130"}]': finished 2024-03-21T13:06:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:31 smithi043 ceph-mon[30062]: osdmap e652: 8 total, 8 up, 8 in 2024-03-21T13:06:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:31 smithi043 ceph-mon[41699]: pgmap v1009: 332 pgs: 332 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:06:32.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:31 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:06:32.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:31 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-130"}]': finished 2024-03-21T13:06:32.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:31 smithi043 ceph-mon[41699]: osdmap e652: 8 total, 8 up, 8 in 2024-03-21T13:06:33.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:32 smithi155 ceph-mon[33846]: osdmap e653: 8 total, 8 up, 8 in 2024-03-21T13:06:33.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:32 smithi155 ceph-mon[33846]: pgmap v1012: 300 pgs: 300 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:06:33.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:32 smithi043 ceph-mon[30062]: osdmap e653: 8 total, 8 up, 8 in 2024-03-21T13:06:33.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:32 smithi043 ceph-mon[30062]: pgmap v1012: 300 pgs: 300 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:06:33.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:32 smithi043 ceph-mon[41699]: osdmap e653: 8 total, 8 up, 8 in 2024-03-21T13:06:33.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:32 smithi043 ceph-mon[41699]: pgmap v1012: 300 pgs: 300 active+clean; 582 KiB data, 680 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:06:34.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:33 smithi155 ceph-mon[33846]: osdmap e654: 8 total, 8 up, 8 in 2024-03-21T13:06:34.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:33 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:34.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:33 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:34.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:33 smithi043 ceph-mon[41699]: osdmap e654: 8 total, 8 up, 8 in 2024-03-21T13:06:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:33 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:33 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:34.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:33 smithi043 ceph-mon[30062]: osdmap e654: 8 total, 8 up, 8 in 2024-03-21T13:06:34.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:33 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:34.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:33 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:35.108 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:34 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:06:35.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:34 smithi155 ceph-mon[33846]: osdmap e655: 8 total, 8 up, 8 in 2024-03-21T13:06:35.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:34 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:35.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:34 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:35.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:34 smithi155 ceph-mon[33846]: pgmap v1015: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:06:35.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:34 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:06:35.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:34 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:06:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:34 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:06:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:34 smithi043 ceph-mon[30062]: osdmap e655: 8 total, 8 up, 8 in 2024-03-21T13:06:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:34 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:34 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:34 smithi043 ceph-mon[30062]: pgmap v1015: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:06:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:34 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:06:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:34 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:06:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:34 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:06:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:34 smithi043 ceph-mon[41699]: osdmap e655: 8 total, 8 up, 8 in 2024-03-21T13:06:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:34 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:34 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:34 smithi043 ceph-mon[41699]: pgmap v1015: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:06:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:34 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:06:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:34 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:06:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:35 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-132", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:06:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:35 smithi155 ceph-mon[33846]: osdmap e656: 8 total, 8 up, 8 in 2024-03-21T13:06:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:35 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-132"}]: dispatch 2024-03-21T13:06:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:35 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-132"}]: dispatch 2024-03-21T13:06:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:35 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-132", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:06:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:35 smithi043 ceph-mon[41699]: osdmap e656: 8 total, 8 up, 8 in 2024-03-21T13:06:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:35 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-132"}]: dispatch 2024-03-21T13:06:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:35 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-132"}]: dispatch 2024-03-21T13:06:36.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:35 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-132", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:06:36.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:35 smithi043 ceph-mon[30062]: osdmap e656: 8 total, 8 up, 8 in 2024-03-21T13:06:36.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:35 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-132"}]: dispatch 2024-03-21T13:06:36.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:35 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-132"}]: dispatch 2024-03-21T13:06:37.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:36 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-132"}]': finished 2024-03-21T13:06:37.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:36 smithi155 ceph-mon[33846]: osdmap e657: 8 total, 8 up, 8 in 2024-03-21T13:06:37.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-132", "mode": "writeback"}]: dispatch 2024-03-21T13:06:37.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:36 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-132", "mode": "writeback"}]: dispatch 2024-03-21T13:06:37.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:36 smithi155 ceph-mon[33846]: pgmap v1018: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:06:37.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:36 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:06:37.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:36 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-132", "mode": "writeback"}]': finished 2024-03-21T13:06:37.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:36 smithi155 ceph-mon[33846]: osdmap e658: 8 total, 8 up, 8 in 2024-03-21T13:06:37.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:36 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-132"}]': finished 2024-03-21T13:06:37.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:36 smithi043 ceph-mon[30062]: osdmap e657: 8 total, 8 up, 8 in 2024-03-21T13:06:37.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-132", "mode": "writeback"}]: dispatch 2024-03-21T13:06:37.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:36 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-132", "mode": "writeback"}]: dispatch 2024-03-21T13:06:37.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:36 smithi043 ceph-mon[30062]: pgmap v1018: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:06:37.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:36 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:06:37.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:36 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-132", "mode": "writeback"}]': finished 2024-03-21T13:06:37.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:36 smithi043 ceph-mon[30062]: osdmap e658: 8 total, 8 up, 8 in 2024-03-21T13:06:37.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:36 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-132"}]': finished 2024-03-21T13:06:37.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:36 smithi043 ceph-mon[41699]: osdmap e657: 8 total, 8 up, 8 in 2024-03-21T13:06:37.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-132", "mode": "writeback"}]: dispatch 2024-03-21T13:06:37.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:36 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-132", "mode": "writeback"}]: dispatch 2024-03-21T13:06:37.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:36 smithi043 ceph-mon[41699]: pgmap v1018: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 681 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:06:37.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:36 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:06:37.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:36 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-132", "mode": "writeback"}]': finished 2024-03-21T13:06:37.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:36 smithi043 ceph-mon[41699]: osdmap e658: 8 total, 8 up, 8 in 2024-03-21T13:06:38.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:06:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:06:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:06:39.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:38 smithi155 ceph-mon[33846]: osdmap e659: 8 total, 8 up, 8 in 2024-03-21T13:06:39.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:38 smithi155 ceph-mon[33846]: pgmap v1021: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 681 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:06:39.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:06:38 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:06:39.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:38 smithi043 ceph-mon[30062]: osdmap e659: 8 total, 8 up, 8 in 2024-03-21T13:06:39.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:38 smithi043 ceph-mon[30062]: pgmap v1021: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 681 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:06:39.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:38 smithi043 ceph-mon[41699]: osdmap e659: 8 total, 8 up, 8 in 2024-03-21T13:06:39.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:38 smithi043 ceph-mon[41699]: pgmap v1021: 332 pgs: 32 creating+peering, 300 active+clean; 582 KiB data, 681 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:06:40.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:39 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:06:40.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:39 smithi155 ceph-mon[33846]: osdmap e660: 8 total, 8 up, 8 in 2024-03-21T13:06:40.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:39 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:40.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:39 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:40.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:39 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:06:40.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:39 smithi043 ceph-mon[30062]: osdmap e660: 8 total, 8 up, 8 in 2024-03-21T13:06:40.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:39 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:40.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:39 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:39 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:06:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:39 smithi043 ceph-mon[41699]: osdmap e660: 8 total, 8 up, 8 in 2024-03-21T13:06:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:39 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:39 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:40 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:06:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:40 smithi155 ceph-mon[33846]: osdmap e661: 8 total, 8 up, 8 in 2024-03-21T13:06:41.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:40 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-132"}]: dispatch 2024-03-21T13:06:41.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:40 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-132"}]: dispatch 2024-03-21T13:06:41.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:40 smithi155 ceph-mon[33846]: pgmap v1024: 332 pgs: 332 active+clean; 582 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:06:41.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:40 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:06:41.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:40 smithi043 ceph-mon[30062]: osdmap e661: 8 total, 8 up, 8 in 2024-03-21T13:06:41.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:40 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-132"}]: dispatch 2024-03-21T13:06:41.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:40 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-132"}]: dispatch 2024-03-21T13:06:41.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:40 smithi043 ceph-mon[30062]: pgmap v1024: 332 pgs: 332 active+clean; 582 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:06:41.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:40 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:06:41.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:40 smithi043 ceph-mon[41699]: osdmap e661: 8 total, 8 up, 8 in 2024-03-21T13:06:41.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:40 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-132"}]: dispatch 2024-03-21T13:06:41.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:40 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-132"}]: dispatch 2024-03-21T13:06:41.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:40 smithi043 ceph-mon[41699]: pgmap v1024: 332 pgs: 332 active+clean; 582 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:06:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:41 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-132"}]': finished 2024-03-21T13:06:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:41 smithi155 ceph-mon[33846]: osdmap e662: 8 total, 8 up, 8 in 2024-03-21T13:06:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:41 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-132"}]': finished 2024-03-21T13:06:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:41 smithi043 ceph-mon[30062]: osdmap e662: 8 total, 8 up, 8 in 2024-03-21T13:06:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:41 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-132"}]': finished 2024-03-21T13:06:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:41 smithi043 ceph-mon[41699]: osdmap e662: 8 total, 8 up, 8 in 2024-03-21T13:06:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:42 smithi155 ceph-mon[33846]: osdmap e663: 8 total, 8 up, 8 in 2024-03-21T13:06:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:42 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:42 smithi155 ceph-mon[33846]: pgmap v1027: 332 pgs: 332 active+clean; 582 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:06:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:42 smithi043 ceph-mon[30062]: osdmap e663: 8 total, 8 up, 8 in 2024-03-21T13:06:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:42 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:42 smithi043 ceph-mon[30062]: pgmap v1027: 332 pgs: 332 active+clean; 582 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:06:43.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:42 smithi043 ceph-mon[41699]: osdmap e663: 8 total, 8 up, 8 in 2024-03-21T13:06:43.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:43.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:42 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:43.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:42 smithi043 ceph-mon[41699]: pgmap v1027: 332 pgs: 332 active+clean; 582 KiB data, 686 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:06:44.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:43 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:06:44.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:43 smithi155 ceph-mon[33846]: osdmap e664: 8 total, 8 up, 8 in 2024-03-21T13:06:44.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-132"}]: dispatch 2024-03-21T13:06:44.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:43 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-132"}]: dispatch 2024-03-21T13:06:44.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:43 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:06:44.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:43 smithi043 ceph-mon[30062]: osdmap e664: 8 total, 8 up, 8 in 2024-03-21T13:06:44.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-132"}]: dispatch 2024-03-21T13:06:44.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:43 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-132"}]: dispatch 2024-03-21T13:06:44.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:43 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:06:44.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:43 smithi043 ceph-mon[41699]: osdmap e664: 8 total, 8 up, 8 in 2024-03-21T13:06:44.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-132"}]: dispatch 2024-03-21T13:06:44.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:43 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-132"}]: dispatch 2024-03-21T13:06:45.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:44 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:06:45.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:44 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-132"}]': finished 2024-03-21T13:06:45.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:44 smithi155 ceph-mon[33846]: osdmap e665: 8 total, 8 up, 8 in 2024-03-21T13:06:45.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:44 smithi155 ceph-mon[33846]: pgmap v1030: 332 pgs: 332 active+clean; 582 KiB data, 695 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-21T13:06:45.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:44 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:06:45.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:44 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-132"}]': finished 2024-03-21T13:06:45.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:44 smithi043 ceph-mon[30062]: osdmap e665: 8 total, 8 up, 8 in 2024-03-21T13:06:45.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:44 smithi043 ceph-mon[30062]: pgmap v1030: 332 pgs: 332 active+clean; 582 KiB data, 695 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-21T13:06:45.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:44 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:06:45.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:44 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-132"}]': finished 2024-03-21T13:06:45.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:44 smithi043 ceph-mon[41699]: osdmap e665: 8 total, 8 up, 8 in 2024-03-21T13:06:45.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:44 smithi043 ceph-mon[41699]: pgmap v1030: 332 pgs: 332 active+clean; 582 KiB data, 695 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-21T13:06:46.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:45 smithi155 ceph-mon[33846]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:06:46.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:45 smithi155 ceph-mon[33846]: osdmap e666: 8 total, 8 up, 8 in 2024-03-21T13:06:46.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:45 smithi043 ceph-mon[30062]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:06:46.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:45 smithi043 ceph-mon[30062]: osdmap e666: 8 total, 8 up, 8 in 2024-03-21T13:06:46.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:45 smithi043 ceph-mon[41699]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:06:46.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:45 smithi043 ceph-mon[41699]: osdmap e666: 8 total, 8 up, 8 in 2024-03-21T13:06:47.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:46 smithi155 ceph-mon[33846]: osdmap e667: 8 total, 8 up, 8 in 2024-03-21T13:06:47.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:47.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:46 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:47.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:46 smithi155 ceph-mon[33846]: pgmap v1033: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 695 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:06:47.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:46 smithi043 ceph-mon[30062]: osdmap e667: 8 total, 8 up, 8 in 2024-03-21T13:06:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:46 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:46 smithi043 ceph-mon[30062]: pgmap v1033: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 695 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:06:47.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:46 smithi043 ceph-mon[41699]: osdmap e667: 8 total, 8 up, 8 in 2024-03-21T13:06:47.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:47.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:46 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:47.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:46 smithi043 ceph-mon[41699]: pgmap v1033: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 695 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:06:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:47 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:06:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:47 smithi155 ceph-mon[33846]: osdmap e668: 8 total, 8 up, 8 in 2024-03-21T13:06:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:48.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:47 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:48.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:47 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-134", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:06:48.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:47 smithi155 ceph-mon[33846]: osdmap e669: 8 total, 8 up, 8 in 2024-03-21T13:06:48.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-134"}]: dispatch 2024-03-21T13:06:48.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:47 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-134"}]: dispatch 2024-03-21T13:06:48.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:06:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:06:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:06:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:47 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:06:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:47 smithi043 ceph-mon[30062]: osdmap e668: 8 total, 8 up, 8 in 2024-03-21T13:06:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:47 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:47 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-134", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:06:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:47 smithi043 ceph-mon[30062]: osdmap e669: 8 total, 8 up, 8 in 2024-03-21T13:06:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-134"}]: dispatch 2024-03-21T13:06:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:47 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-134"}]: dispatch 2024-03-21T13:06:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:47 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:06:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:47 smithi043 ceph-mon[41699]: osdmap e668: 8 total, 8 up, 8 in 2024-03-21T13:06:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:47 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:47 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-134", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:06:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:47 smithi043 ceph-mon[41699]: osdmap e669: 8 total, 8 up, 8 in 2024-03-21T13:06:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-134"}]: dispatch 2024-03-21T13:06:48.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:47 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-134"}]: dispatch 2024-03-21T13:06:49.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:06:48 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:06:49.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:49 smithi155 ceph-mon[33846]: pgmap v1036: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 695 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:06:49.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:49 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-134"}]': finished 2024-03-21T13:06:49.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:49 smithi155 ceph-mon[33846]: osdmap e670: 8 total, 8 up, 8 in 2024-03-21T13:06:49.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:49 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-134", "mode": "writeback"}]: dispatch 2024-03-21T13:06:49.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:49 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-134", "mode": "writeback"}]: dispatch 2024-03-21T13:06:49.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:49 smithi043 ceph-mon[41699]: pgmap v1036: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 695 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:06:49.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:49 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-134"}]': finished 2024-03-21T13:06:49.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:49 smithi043 ceph-mon[41699]: osdmap e670: 8 total, 8 up, 8 in 2024-03-21T13:06:49.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:49 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-134", "mode": "writeback"}]: dispatch 2024-03-21T13:06:49.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:49 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-134", "mode": "writeback"}]: dispatch 2024-03-21T13:06:49.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:49 smithi043 ceph-mon[30062]: pgmap v1036: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 695 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:06:49.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:49 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-134"}]': finished 2024-03-21T13:06:49.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:49 smithi043 ceph-mon[30062]: osdmap e670: 8 total, 8 up, 8 in 2024-03-21T13:06:49.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:49 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-134", "mode": "writeback"}]: dispatch 2024-03-21T13:06:49.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:49 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-134", "mode": "writeback"}]: dispatch 2024-03-21T13:06:50.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:50 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:06:50.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:50 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:06:50.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:50 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-134", "mode": "writeback"}]': finished 2024-03-21T13:06:50.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:50 smithi155 ceph-mon[33846]: osdmap e671: 8 total, 8 up, 8 in 2024-03-21T13:06:50.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:50 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:06:50.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:50 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:50.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:50 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:50.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:50 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:06:50.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:50 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:06:50.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:50 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-134", "mode": "writeback"}]': finished 2024-03-21T13:06:50.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:50 smithi043 ceph-mon[30062]: osdmap e671: 8 total, 8 up, 8 in 2024-03-21T13:06:50.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:50 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:06:50.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:50 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:50.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:50 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:50.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:50 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:06:50.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:50 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:06:50.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:50 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-134", "mode": "writeback"}]': finished 2024-03-21T13:06:50.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:50 smithi043 ceph-mon[41699]: osdmap e671: 8 total, 8 up, 8 in 2024-03-21T13:06:50.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:50 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:06:50.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:50 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:50.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:50 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:51.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:51 smithi155 ceph-mon[33846]: pgmap v1039: 332 pgs: 332 active+clean; 582 KiB data, 699 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:06:51.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:51 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:06:51.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:51 smithi155 ceph-mon[33846]: osdmap e672: 8 total, 8 up, 8 in 2024-03-21T13:06:51.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:51 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-134"}]: dispatch 2024-03-21T13:06:51.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:51 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-134"}]: dispatch 2024-03-21T13:06:51.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:51 smithi043 ceph-mon[30062]: pgmap v1039: 332 pgs: 332 active+clean; 582 KiB data, 699 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:06:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:51 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:06:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:51 smithi043 ceph-mon[30062]: osdmap e672: 8 total, 8 up, 8 in 2024-03-21T13:06:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:51 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-134"}]: dispatch 2024-03-21T13:06:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:51 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-134"}]: dispatch 2024-03-21T13:06:51.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:51 smithi043 ceph-mon[41699]: pgmap v1039: 332 pgs: 332 active+clean; 582 KiB data, 699 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:06:51.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:51 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:06:51.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:51 smithi043 ceph-mon[41699]: osdmap e672: 8 total, 8 up, 8 in 2024-03-21T13:06:51.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:51 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-134"}]: dispatch 2024-03-21T13:06:51.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:51 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-134"}]: dispatch 2024-03-21T13:06:52.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:52 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:06:52.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:52 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-134"}]': finished 2024-03-21T13:06:52.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:52 smithi155 ceph-mon[33846]: osdmap e673: 8 total, 8 up, 8 in 2024-03-21T13:06:52.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:52 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:06:52.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:52 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:06:52.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:52 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-134"}]': finished 2024-03-21T13:06:52.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:52 smithi043 ceph-mon[30062]: osdmap e673: 8 total, 8 up, 8 in 2024-03-21T13:06:52.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:52 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:06:52.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:52 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:06:52.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:52 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-134"}]': finished 2024-03-21T13:06:52.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:52 smithi043 ceph-mon[41699]: osdmap e673: 8 total, 8 up, 8 in 2024-03-21T13:06:52.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:52 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:06:53.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:53 smithi155 ceph-mon[33846]: pgmap v1042: 332 pgs: 332 active+clean; 582 KiB data, 699 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:06:53.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:53 smithi155 ceph-mon[33846]: osdmap e674: 8 total, 8 up, 8 in 2024-03-21T13:06:53.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:53 smithi043 ceph-mon[30062]: pgmap v1042: 332 pgs: 332 active+clean; 582 KiB data, 699 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:06:53.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:53 smithi043 ceph-mon[30062]: osdmap e674: 8 total, 8 up, 8 in 2024-03-21T13:06:53.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:53 smithi043 ceph-mon[41699]: pgmap v1042: 332 pgs: 332 active+clean; 582 KiB data, 699 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:06:53.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:53 smithi043 ceph-mon[41699]: osdmap e674: 8 total, 8 up, 8 in 2024-03-21T13:06:54.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:54 smithi043 ceph-mon[30062]: osdmap e675: 8 total, 8 up, 8 in 2024-03-21T13:06:54.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:54 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:54.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:54 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:54.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:54 smithi043 ceph-mon[41699]: osdmap e675: 8 total, 8 up, 8 in 2024-03-21T13:06:54.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:54 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:54.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:54 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:55.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:54 smithi155 ceph-mon[33846]: osdmap e675: 8 total, 8 up, 8 in 2024-03-21T13:06:55.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:54 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:55.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:54 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:06:55.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:55 smithi043 ceph-mon[30062]: pgmap v1045: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:06:55.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:55 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:06:55.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:55 smithi043 ceph-mon[30062]: osdmap e676: 8 total, 8 up, 8 in 2024-03-21T13:06:55.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:55 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:55.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:55 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:55.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:55 smithi043 ceph-mon[41699]: pgmap v1045: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:06:55.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:55 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:06:55.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:55 smithi043 ceph-mon[41699]: osdmap e676: 8 total, 8 up, 8 in 2024-03-21T13:06:55.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:55 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:55.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:55 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:55 smithi155 ceph-mon[33846]: pgmap v1045: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:06:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:55 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:06:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:55 smithi155 ceph-mon[33846]: osdmap e676: 8 total, 8 up, 8 in 2024-03-21T13:06:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:55 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:55 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:06:56.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:56 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-136", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:06:56.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:56 smithi043 ceph-mon[30062]: osdmap e677: 8 total, 8 up, 8 in 2024-03-21T13:06:56.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:56 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-136"}]: dispatch 2024-03-21T13:06:56.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:56 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-136"}]: dispatch 2024-03-21T13:06:56.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:56 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-136"}]': finished 2024-03-21T13:06:56.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:56 smithi043 ceph-mon[30062]: osdmap e678: 8 total, 8 up, 8 in 2024-03-21T13:06:56.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:56 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-136", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:06:56.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:56 smithi043 ceph-mon[41699]: osdmap e677: 8 total, 8 up, 8 in 2024-03-21T13:06:56.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:56 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-136"}]: dispatch 2024-03-21T13:06:56.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:56 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-136"}]: dispatch 2024-03-21T13:06:56.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:56 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-136"}]': finished 2024-03-21T13:06:56.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:56 smithi043 ceph-mon[41699]: osdmap e678: 8 total, 8 up, 8 in 2024-03-21T13:06:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:56 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-136", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:06:57.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:56 smithi155 ceph-mon[33846]: osdmap e677: 8 total, 8 up, 8 in 2024-03-21T13:06:57.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:56 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-136"}]: dispatch 2024-03-21T13:06:57.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:56 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-136"}]: dispatch 2024-03-21T13:06:57.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:56 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-136"}]': finished 2024-03-21T13:06:57.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:56 smithi155 ceph-mon[33846]: osdmap e678: 8 total, 8 up, 8 in 2024-03-21T13:06:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:57 smithi043 ceph-mon[30062]: pgmap v1048: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:06:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:57 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-136", "mode": "writeback"}]: dispatch 2024-03-21T13:06:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:57 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-136", "mode": "writeback"}]: dispatch 2024-03-21T13:06:57.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:57 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:06:57.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:57 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-136", "mode": "writeback"}]': finished 2024-03-21T13:06:57.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:57 smithi043 ceph-mon[30062]: osdmap e679: 8 total, 8 up, 8 in 2024-03-21T13:06:57.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:57 smithi043 ceph-mon[41699]: pgmap v1048: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:06:57.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:57 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-136", "mode": "writeback"}]: dispatch 2024-03-21T13:06:57.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:57 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-136", "mode": "writeback"}]: dispatch 2024-03-21T13:06:57.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:57 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:06:57.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:57 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-136", "mode": "writeback"}]': finished 2024-03-21T13:06:57.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:57 smithi043 ceph-mon[41699]: osdmap e679: 8 total, 8 up, 8 in 2024-03-21T13:06:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:57 smithi155 ceph-mon[33846]: pgmap v1048: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:06:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:57 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-136", "mode": "writeback"}]: dispatch 2024-03-21T13:06:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:57 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-136", "mode": "writeback"}]: dispatch 2024-03-21T13:06:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:57 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:06:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:57 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-136", "mode": "writeback"}]': finished 2024-03-21T13:06:58.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:57 smithi155 ceph-mon[33846]: osdmap e679: 8 total, 8 up, 8 in 2024-03-21T13:06:58.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:06:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:06:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:06:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:58 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:58 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:58.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:58 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:58.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:58 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:59.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:06:58 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:06:59.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:58 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:06:59.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:58 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:07:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:59 smithi155 ceph-mon[33846]: pgmap v1051: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 700 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:07:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:59 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:07:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:59 smithi155 ceph-mon[33846]: osdmap e680: 8 total, 8 up, 8 in 2024-03-21T13:07:00.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:59 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-136"}]: dispatch 2024-03-21T13:07:00.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:06:59 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-136"}]: dispatch 2024-03-21T13:07:00.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:59 smithi043 ceph-mon[30062]: pgmap v1051: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 700 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:07:00.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:59 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:07:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:59 smithi043 ceph-mon[30062]: osdmap e680: 8 total, 8 up, 8 in 2024-03-21T13:07:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:59 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-136"}]: dispatch 2024-03-21T13:07:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:06:59 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-136"}]: dispatch 2024-03-21T13:07:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:59 smithi043 ceph-mon[41699]: pgmap v1051: 332 pgs: 32 unknown, 300 active+clean; 582 KiB data, 700 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:07:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:59 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:07:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:59 smithi043 ceph-mon[41699]: osdmap e680: 8 total, 8 up, 8 in 2024-03-21T13:07:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:59 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-136"}]: dispatch 2024-03-21T13:07:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:06:59 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-136"}]: dispatch 2024-03-21T13:07:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:00 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:07:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:00 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:07:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:00 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-136"}]': finished 2024-03-21T13:07:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:00 smithi155 ceph-mon[33846]: osdmap e681: 8 total, 8 up, 8 in 2024-03-21T13:07:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:00 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:07:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:00 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:07:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:00 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-136"}]': finished 2024-03-21T13:07:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:00 smithi043 ceph-mon[30062]: osdmap e681: 8 total, 8 up, 8 in 2024-03-21T13:07:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:00 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:07:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:00 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:07:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:00 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-136"}]': finished 2024-03-21T13:07:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:00 smithi043 ceph-mon[41699]: osdmap e681: 8 total, 8 up, 8 in 2024-03-21T13:07:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:01 smithi155 ceph-mon[33846]: pgmap v1054: 332 pgs: 332 active+clean; 4.4 MiB data, 692 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-03-21T13:07:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:01 smithi155 ceph-mon[33846]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:07:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:01 smithi155 ceph-mon[33846]: osdmap e682: 8 total, 8 up, 8 in 2024-03-21T13:07:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:01 smithi043 ceph-mon[30062]: pgmap v1054: 332 pgs: 332 active+clean; 4.4 MiB data, 692 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-03-21T13:07:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:01 smithi043 ceph-mon[30062]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:07:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:01 smithi043 ceph-mon[30062]: osdmap e682: 8 total, 8 up, 8 in 2024-03-21T13:07:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:01 smithi043 ceph-mon[41699]: pgmap v1054: 332 pgs: 332 active+clean; 4.4 MiB data, 692 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 1 op/s 2024-03-21T13:07:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:01 smithi043 ceph-mon[41699]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:07:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:01 smithi043 ceph-mon[41699]: osdmap e682: 8 total, 8 up, 8 in 2024-03-21T13:07:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:02 smithi155 ceph-mon[33846]: osdmap e683: 8 total, 8 up, 8 in 2024-03-21T13:07:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:02 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:07:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:02 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:07:03.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:02 smithi043 ceph-mon[30062]: osdmap e683: 8 total, 8 up, 8 in 2024-03-21T13:07:03.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:02 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:07:03.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:02 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:07:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:02 smithi043 ceph-mon[41699]: osdmap e683: 8 total, 8 up, 8 in 2024-03-21T13:07:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:02 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:07:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:02 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:07:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:03 smithi155 ceph-mon[33846]: pgmap v1057: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 692 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-21T13:07:04.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:03 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:07:04.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:03 smithi155 ceph-mon[33846]: osdmap e684: 8 total, 8 up, 8 in 2024-03-21T13:07:04.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:03 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:07:04.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:03 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:07:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:03 smithi043 ceph-mon[30062]: pgmap v1057: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 692 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-21T13:07:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:03 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:07:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:03 smithi043 ceph-mon[30062]: osdmap e684: 8 total, 8 up, 8 in 2024-03-21T13:07:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:03 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:07:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:03 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:07:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:03 smithi043 ceph-mon[41699]: pgmap v1057: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 692 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-21T13:07:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:03 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:07:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:03 smithi043 ceph-mon[41699]: osdmap e684: 8 total, 8 up, 8 in 2024-03-21T13:07:04.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:03 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:07:04.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:03 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:07:05.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:04 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-138", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:07:05.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:04 smithi155 ceph-mon[33846]: osdmap e685: 8 total, 8 up, 8 in 2024-03-21T13:07:05.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:07:05.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:04 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:07:05.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:04 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:07:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:04 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-138", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:07:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:04 smithi043 ceph-mon[30062]: osdmap e685: 8 total, 8 up, 8 in 2024-03-21T13:07:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:07:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:04 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:07:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:04 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:07:05.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:04 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-138", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:07:05.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:04 smithi043 ceph-mon[41699]: osdmap e685: 8 total, 8 up, 8 in 2024-03-21T13:07:05.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:07:05.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:04 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:07:05.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:04 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:07:05.653 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ]c 2024-03-21T13:07:05.653 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.653 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.653 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.653 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.654 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.655 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.656 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.657 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.658 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.659 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.660 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.660 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.660 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.660 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.660 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.660 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.660 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:05.660 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:07:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:05 smithi155 ceph-mon[33846]: pgmap v1060: 332 pgs: 332 active+clean; 4.4 MiB data, 699 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:07:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:05 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:07:06.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:05 smithi155 ceph-mon[33846]: osdmap e686: 8 total, 8 up, 8 in 2024-03-21T13:07:06.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:05 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:07:06.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:05 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:07:06.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:05 smithi043 ceph-mon[30062]: pgmap v1060: 332 pgs: 332 active+clean; 4.4 MiB data, 699 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:07:06.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:05 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:07:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:05 smithi043 ceph-mon[30062]: osdmap e686: 8 total, 8 up, 8 in 2024-03-21T13:07:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:05 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:07:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:05 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:07:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:05 smithi043 ceph-mon[41699]: pgmap v1060: 332 pgs: 332 active+clean; 4.4 MiB data, 699 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:07:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:05 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:07:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:05 smithi043 ceph-mon[41699]: osdmap e686: 8 total, 8 up, 8 in 2024-03-21T13:07:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:05 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:07:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:05 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:07:07.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:06 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:07:07.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:06 smithi155 ceph-mon[33846]: osdmap e687: 8 total, 8 up, 8 in 2024-03-21T13:07:07.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:06 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:07:07.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:06 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:07:07.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:06 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:07:07.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:06 smithi043 ceph-mon[30062]: osdmap e687: 8 total, 8 up, 8 in 2024-03-21T13:07:07.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:06 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:07:07.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:06 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:07:07.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:06 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:07:07.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:06 smithi043 ceph-mon[41699]: osdmap e687: 8 total, 8 up, 8 in 2024-03-21T13:07:07.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:06 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:07:07.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:06 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-21T13:07:08.134 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:07 smithi043 ceph-mon[30062]: pgmap v1063: 332 pgs: 332 active+clean; 4.4 MiB data, 699 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:07:08.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:07 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-21T13:07:08.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:07 smithi043 ceph-mon[30062]: osdmap e688: 8 total, 8 up, 8 in 2024-03-21T13:07:08.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:07 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:07:08.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:07 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:07:08.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:07 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:07:08.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:07 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:07:08.135 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:07:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:07:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:07:08.135 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:07 smithi043 ceph-mon[41699]: pgmap v1063: 332 pgs: 332 active+clean; 4.4 MiB data, 699 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:07:08.136 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:07 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-21T13:07:08.136 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:07 smithi043 ceph-mon[41699]: osdmap e688: 8 total, 8 up, 8 in 2024-03-21T13:07:08.136 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:07 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:07:08.136 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:07 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:07:08.136 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:07 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:07:08.136 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:07 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:07:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:07 smithi155 ceph-mon[33846]: pgmap v1063: 332 pgs: 332 active+clean; 4.4 MiB data, 699 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:07:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:07 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-21T13:07:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:07 smithi155 ceph-mon[33846]: osdmap e688: 8 total, 8 up, 8 in 2024-03-21T13:07:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:07 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:07:08.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:07 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:07:08.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:07 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:07:08.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:07 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:07:09.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:07:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:07:09.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:08 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:07:09.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:08 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:07:09.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:08 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-138"}]: dispatch 2024-03-21T13:07:09.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:08 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-138"}]: dispatch 2024-03-21T13:07:09.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:08 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:07:09.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:08 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:07:09.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:08 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:07:09.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:08 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:07:09.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:08 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:07:09.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:08 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-138"}]: dispatch 2024-03-21T13:07:09.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:08 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-138"}]: dispatch 2024-03-21T13:07:09.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:08 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:07:09.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:08 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:07:09.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:08 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:07:09.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:08 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:07:09.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:08 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:07:09.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:08 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-138"}]: dispatch 2024-03-21T13:07:09.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:08 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-138"}]: dispatch 2024-03-21T13:07:09.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:08 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:07:09.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:08 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:07:09.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:08 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:07:10.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:09 smithi155 ceph-mon[33846]: pgmap v1065: 332 pgs: 332 active+clean; 4.4 MiB data, 699 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:07:10.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:09 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-138"}]': finished 2024-03-21T13:07:10.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:09 smithi155 ceph-mon[33846]: osdmap e689: 8 total, 8 up, 8 in 2024-03-21T13:07:10.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:09 smithi043 ceph-mon[30062]: pgmap v1065: 332 pgs: 332 active+clean; 4.4 MiB data, 699 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:07:10.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:09 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-138"}]': finished 2024-03-21T13:07:10.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:09 smithi043 ceph-mon[30062]: osdmap e689: 8 total, 8 up, 8 in 2024-03-21T13:07:10.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:09 smithi043 ceph-mon[41699]: pgmap v1065: 332 pgs: 332 active+clean; 4.4 MiB data, 699 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:07:10.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:09 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-138"}]': finished 2024-03-21T13:07:10.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:09 smithi043 ceph-mon[41699]: osdmap e689: 8 total, 8 up, 8 in 2024-03-21T13:07:11.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:10 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:07:11.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:10 smithi155 ceph-mon[33846]: osdmap e690: 8 total, 8 up, 8 in 2024-03-21T13:07:11.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:10 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:07:11.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:10 smithi043 ceph-mon[30062]: osdmap e690: 8 total, 8 up, 8 in 2024-03-21T13:07:11.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:10 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:07:11.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:10 smithi043 ceph-mon[41699]: osdmap e690: 8 total, 8 up, 8 in 2024-03-21T13:07:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:11 smithi155 ceph-mon[33846]: pgmap v1068: 300 pgs: 300 active+clean; 4.4 MiB data, 699 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-21T13:07:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:11 smithi155 ceph-mon[33846]: osdmap e691: 8 total, 8 up, 8 in 2024-03-21T13:07:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:11 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:07:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:11 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:07:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:11 smithi043 ceph-mon[30062]: pgmap v1068: 300 pgs: 300 active+clean; 4.4 MiB data, 699 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-21T13:07:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:11 smithi043 ceph-mon[30062]: osdmap e691: 8 total, 8 up, 8 in 2024-03-21T13:07:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:11 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:07:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:11 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:07:12.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:11 smithi043 ceph-mon[41699]: pgmap v1068: 300 pgs: 300 active+clean; 4.4 MiB data, 699 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-21T13:07:12.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:11 smithi043 ceph-mon[41699]: osdmap e691: 8 total, 8 up, 8 in 2024-03-21T13:07:12.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:11 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:07:12.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:11 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:07:13.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:12 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:07:13.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:12 smithi155 ceph-mon[33846]: osdmap e692: 8 total, 8 up, 8 in 2024-03-21T13:07:13.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:12 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:07:13.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:12 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:07:13.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:12 smithi155 ceph-mon[33846]: pgmap v1071: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 699 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:07:13.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:12 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:07:13.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:12 smithi043 ceph-mon[30062]: osdmap e692: 8 total, 8 up, 8 in 2024-03-21T13:07:13.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:12 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:07:13.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:12 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:07:13.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:12 smithi043 ceph-mon[30062]: pgmap v1071: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 699 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:07:13.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:12 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:07:13.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:12 smithi043 ceph-mon[41699]: osdmap e692: 8 total, 8 up, 8 in 2024-03-21T13:07:13.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:12 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:07:13.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:12 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:07:13.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:12 smithi043 ceph-mon[41699]: pgmap v1071: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 699 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:07:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:13 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-140", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:07:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:13 smithi155 ceph-mon[33846]: osdmap e693: 8 total, 8 up, 8 in 2024-03-21T13:07:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:13 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_count","val": "3"}]: dispatch 2024-03-21T13:07:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:13 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_count","val": "3"}]: dispatch 2024-03-21T13:07:14.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:13 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-140", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:07:14.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:13 smithi043 ceph-mon[30062]: osdmap e693: 8 total, 8 up, 8 in 2024-03-21T13:07:14.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:13 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_count","val": "3"}]: dispatch 2024-03-21T13:07:14.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:13 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_count","val": "3"}]: dispatch 2024-03-21T13:07:14.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:13 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-140", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:07:14.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:13 smithi043 ceph-mon[41699]: osdmap e693: 8 total, 8 up, 8 in 2024-03-21T13:07:14.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:13 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_count","val": "3"}]: dispatch 2024-03-21T13:07:14.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:13 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_count","val": "3"}]: dispatch 2024-03-21T13:07:15.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:14 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_count","val": "3"}]': finished 2024-03-21T13:07:15.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:14 smithi155 ceph-mon[33846]: osdmap e694: 8 total, 8 up, 8 in 2024-03-21T13:07:15.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:14 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_period","val": "3"}]: dispatch 2024-03-21T13:07:15.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:14 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_period","val": "3"}]: dispatch 2024-03-21T13:07:15.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:14 smithi155 ceph-mon[33846]: pgmap v1074: 332 pgs: 332 active+clean; 4.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:07:15.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:14 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_count","val": "3"}]': finished 2024-03-21T13:07:15.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:14 smithi043 ceph-mon[30062]: osdmap e694: 8 total, 8 up, 8 in 2024-03-21T13:07:15.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:14 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_period","val": "3"}]: dispatch 2024-03-21T13:07:15.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:14 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_period","val": "3"}]: dispatch 2024-03-21T13:07:15.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:14 smithi043 ceph-mon[30062]: pgmap v1074: 332 pgs: 332 active+clean; 4.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:07:15.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:14 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_count","val": "3"}]': finished 2024-03-21T13:07:15.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:14 smithi043 ceph-mon[41699]: osdmap e694: 8 total, 8 up, 8 in 2024-03-21T13:07:15.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:14 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_period","val": "3"}]: dispatch 2024-03-21T13:07:15.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:14 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_period","val": "3"}]: dispatch 2024-03-21T13:07:15.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:14 smithi043 ceph-mon[41699]: pgmap v1074: 332 pgs: 332 active+clean; 4.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:07:16.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:15 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_period","val": "3"}]': finished 2024-03-21T13:07:16.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:15 smithi155 ceph-mon[33846]: osdmap e695: 8 total, 8 up, 8 in 2024-03-21T13:07:16.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:15 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:07:16.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:15 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:07:16.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:15 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_period","val": "3"}]': finished 2024-03-21T13:07:16.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:15 smithi043 ceph-mon[30062]: osdmap e695: 8 total, 8 up, 8 in 2024-03-21T13:07:16.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:15 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:07:16.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:15 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:07:16.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:15 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_period","val": "3"}]': finished 2024-03-21T13:07:16.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:15 smithi043 ceph-mon[41699]: osdmap e695: 8 total, 8 up, 8 in 2024-03-21T13:07:16.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:15 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:07:16.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:15 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:07:17.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:16 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:07:17.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:16 smithi155 ceph-mon[33846]: osdmap e696: 8 total, 8 up, 8 in 2024-03-21T13:07:17.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:16 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-21T13:07:17.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:16 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-21T13:07:17.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:16 smithi155 ceph-mon[33846]: pgmap v1077: 332 pgs: 332 active+clean; 4.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:07:17.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:16 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:07:17.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:16 smithi043 ceph-mon[30062]: osdmap e696: 8 total, 8 up, 8 in 2024-03-21T13:07:17.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:16 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-21T13:07:17.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:16 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-21T13:07:17.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:16 smithi043 ceph-mon[30062]: pgmap v1077: 332 pgs: 332 active+clean; 4.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:07:17.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:16 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:07:17.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:16 smithi043 ceph-mon[41699]: osdmap e696: 8 total, 8 up, 8 in 2024-03-21T13:07:17.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:16 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-21T13:07:17.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:16 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-21T13:07:17.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:16 smithi043 ceph-mon[41699]: pgmap v1077: 332 pgs: 332 active+clean; 4.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:07:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:17 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-21T13:07:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:17 smithi155 ceph-mon[33846]: osdmap e697: 8 total, 8 up, 8 in 2024-03-21T13:07:18.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:17 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-21T13:07:18.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:17 smithi043 ceph-mon[30062]: osdmap e697: 8 total, 8 up, 8 in 2024-03-21T13:07:18.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:07:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:07:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:07:18.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:17 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-21T13:07:18.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:17 smithi043 ceph-mon[41699]: osdmap e697: 8 total, 8 up, 8 in 2024-03-21T13:07:19.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:18 smithi155 ceph-mon[33846]: pgmap v1079: 332 pgs: 332 active+clean; 4.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:07:19.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:07:18 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:07:19.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:18 smithi043 ceph-mon[30062]: pgmap v1079: 332 pgs: 332 active+clean; 4.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:07:19.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:18 smithi043 ceph-mon[41699]: pgmap v1079: 332 pgs: 332 active+clean; 4.4 MiB data, 700 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:07:20.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:19 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:07:20.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:19 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:07:20.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:19 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:07:20.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:19 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:07:20.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:19 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:07:20.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:19 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:07:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:20 smithi155 ceph-mon[33846]: pgmap v1080: 332 pgs: 332 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 2.7 KiB/s wr, 1 op/s 2024-03-21T13:07:21.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:20 smithi043 ceph-mon[30062]: pgmap v1080: 332 pgs: 332 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 2.7 KiB/s wr, 1 op/s 2024-03-21T13:07:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:20 smithi043 ceph-mon[41699]: pgmap v1080: 332 pgs: 332 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 2.7 KiB/s wr, 1 op/s 2024-03-21T13:07:23.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:23 smithi043 ceph-mon[30062]: pgmap v1081: 332 pgs: 332 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 698 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-03-21T13:07:23.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:23 smithi043 ceph-mon[41699]: pgmap v1081: 332 pgs: 332 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 698 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-03-21T13:07:23.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:23 smithi155 ceph-mon[33846]: pgmap v1081: 332 pgs: 332 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 698 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-03-21T13:07:25.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:25 smithi043 ceph-mon[30062]: pgmap v1082: 332 pgs: 332 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.0 KiB/s wr, 2 op/s 2024-03-21T13:07:25.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:25 smithi043 ceph-mon[41699]: pgmap v1082: 332 pgs: 332 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.0 KiB/s wr, 2 op/s 2024-03-21T13:07:25.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:25 smithi155 ceph-mon[33846]: pgmap v1082: 332 pgs: 332 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 7.0 KiB/s wr, 2 op/s 2024-03-21T13:07:27.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:27 smithi043 ceph-mon[30062]: pgmap v1083: 332 pgs: 332 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 5.8 KiB/s wr, 1 op/s 2024-03-21T13:07:27.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:27 smithi043 ceph-mon[41699]: pgmap v1083: 332 pgs: 332 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 5.8 KiB/s wr, 1 op/s 2024-03-21T13:07:27.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:27 smithi155 ceph-mon[33846]: pgmap v1083: 332 pgs: 332 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 5.8 KiB/s wr, 1 op/s 2024-03-21T13:07:28.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:07:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:07:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:07:29.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:07:28 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:07:29.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:29 smithi043 ceph-mon[30062]: pgmap v1084: 332 pgs: 332 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 905 B/s rd, 5.1 KiB/s wr, 1 op/s 2024-03-21T13:07:29.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:29 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:07:29.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:29 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:07:29.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:29 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-140"}]: dispatch 2024-03-21T13:07:29.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:29 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-140"}]: dispatch 2024-03-21T13:07:29.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:29 smithi043 ceph-mon[41699]: pgmap v1084: 332 pgs: 332 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 905 B/s rd, 5.1 KiB/s wr, 1 op/s 2024-03-21T13:07:29.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:29 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:07:29.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:29 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:07:29.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:29 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-140"}]: dispatch 2024-03-21T13:07:29.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:29 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-140"}]: dispatch 2024-03-21T13:07:29.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:29 smithi155 ceph-mon[33846]: pgmap v1084: 332 pgs: 332 active+clean; 4.4 MiB data, 718 MiB used, 715 GiB / 715 GiB avail; 905 B/s rd, 5.1 KiB/s wr, 1 op/s 2024-03-21T13:07:29.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:29 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:07:29.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:29 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:07:29.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:29 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-140"}]: dispatch 2024-03-21T13:07:29.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:29 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-140"}]: dispatch 2024-03-21T13:07:30.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:30 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:07:30.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:30 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-140"}]': finished 2024-03-21T13:07:30.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:30 smithi043 ceph-mon[30062]: osdmap e698: 8 total, 8 up, 8 in 2024-03-21T13:07:30.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:30 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:07:30.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:30 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-140"}]': finished 2024-03-21T13:07:30.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:30 smithi043 ceph-mon[41699]: osdmap e698: 8 total, 8 up, 8 in 2024-03-21T13:07:30.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:30 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:07:30.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:30 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-140"}]': finished 2024-03-21T13:07:30.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:30 smithi155 ceph-mon[33846]: osdmap e698: 8 total, 8 up, 8 in 2024-03-21T13:07:31.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:31 smithi043 ceph-mon[30062]: pgmap v1086: 332 pgs: 332 active+clean; 4.5 MiB data, 719 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-03-21T13:07:31.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:31 smithi043 ceph-mon[30062]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:07:31.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:31 smithi043 ceph-mon[30062]: osdmap e699: 8 total, 8 up, 8 in 2024-03-21T13:07:31.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:31 smithi043 ceph-mon[41699]: pgmap v1086: 332 pgs: 332 active+clean; 4.5 MiB data, 719 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-03-21T13:07:31.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:31 smithi043 ceph-mon[41699]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:07:31.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:31 smithi043 ceph-mon[41699]: osdmap e699: 8 total, 8 up, 8 in 2024-03-21T13:07:31.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:31 smithi155 ceph-mon[33846]: pgmap v1086: 332 pgs: 332 active+clean; 4.5 MiB data, 719 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-03-21T13:07:31.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:31 smithi155 ceph-mon[33846]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:07:31.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:31 smithi155 ceph-mon[33846]: osdmap e699: 8 total, 8 up, 8 in 2024-03-21T13:07:32.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:32 smithi155 ceph-mon[33846]: osdmap e700: 8 total, 8 up, 8 in 2024-03-21T13:07:32.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:32 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:07:32.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:32 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:07:32.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:32 smithi043 ceph-mon[30062]: osdmap e700: 8 total, 8 up, 8 in 2024-03-21T13:07:32.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:32 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:07:32.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:32 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:07:32.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:32 smithi043 ceph-mon[41699]: osdmap e700: 8 total, 8 up, 8 in 2024-03-21T13:07:32.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:32 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:07:32.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:32 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:07:33.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:33 smithi155 ceph-mon[33846]: pgmap v1089: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 719 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:07:33.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:33 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:07:33.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:33 smithi155 ceph-mon[33846]: osdmap e701: 8 total, 8 up, 8 in 2024-03-21T13:07:33.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:33 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:07:33.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:33 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:07:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:33 smithi043 ceph-mon[30062]: pgmap v1089: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 719 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:07:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:33 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:07:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:33 smithi043 ceph-mon[30062]: osdmap e701: 8 total, 8 up, 8 in 2024-03-21T13:07:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:33 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:07:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:33 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:07:33.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:33 smithi043 ceph-mon[41699]: pgmap v1089: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 719 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:07:33.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:33 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:07:33.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:33 smithi043 ceph-mon[41699]: osdmap e701: 8 total, 8 up, 8 in 2024-03-21T13:07:33.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:33 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:07:33.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:33 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:07:34.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:34 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-142", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:07:34.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:34 smithi155 ceph-mon[33846]: osdmap e702: 8 total, 8 up, 8 in 2024-03-21T13:07:34.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:34 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-142"}]: dispatch 2024-03-21T13:07:34.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:34 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-142"}]: dispatch 2024-03-21T13:07:34.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:34 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-142", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:07:34.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:34 smithi043 ceph-mon[30062]: osdmap e702: 8 total, 8 up, 8 in 2024-03-21T13:07:34.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:34 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-142"}]: dispatch 2024-03-21T13:07:34.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:34 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-142"}]: dispatch 2024-03-21T13:07:34.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:34 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-142", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:07:34.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:34 smithi043 ceph-mon[41699]: osdmap e702: 8 total, 8 up, 8 in 2024-03-21T13:07:34.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:34 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-142"}]: dispatch 2024-03-21T13:07:34.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:34 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-142"}]: dispatch 2024-03-21T13:07:35.608 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:35 smithi155 ceph-mon[33846]: pgmap v1092: 332 pgs: 332 active+clean; 4.4 MiB data, 719 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:07:35.609 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:35 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-142"}]': finished 2024-03-21T13:07:35.609 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:35 smithi155 ceph-mon[33846]: osdmap e703: 8 total, 8 up, 8 in 2024-03-21T13:07:35.609 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:35 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-142", "mode": "writeback"}]: dispatch 2024-03-21T13:07:35.609 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:35 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-142", "mode": "writeback"}]: dispatch 2024-03-21T13:07:35.609 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:07:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:35 smithi043 ceph-mon[30062]: pgmap v1092: 332 pgs: 332 active+clean; 4.4 MiB data, 719 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:07:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:35 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-142"}]': finished 2024-03-21T13:07:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:35 smithi043 ceph-mon[30062]: osdmap e703: 8 total, 8 up, 8 in 2024-03-21T13:07:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:35 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-142", "mode": "writeback"}]: dispatch 2024-03-21T13:07:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:35 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-142", "mode": "writeback"}]: dispatch 2024-03-21T13:07:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:07:35.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:35 smithi043 ceph-mon[41699]: pgmap v1092: 332 pgs: 332 active+clean; 4.4 MiB data, 719 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-21T13:07:35.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:35 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-142"}]': finished 2024-03-21T13:07:35.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:35 smithi043 ceph-mon[41699]: osdmap e703: 8 total, 8 up, 8 in 2024-03-21T13:07:35.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:35 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-142", "mode": "writeback"}]: dispatch 2024-03-21T13:07:35.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:35 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-142", "mode": "writeback"}]: dispatch 2024-03-21T13:07:35.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:07:36.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:36 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:07:36.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:36 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-142", "mode": "writeback"}]': finished 2024-03-21T13:07:36.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:36 smithi155 ceph-mon[33846]: osdmap e704: 8 total, 8 up, 8 in 2024-03-21T13:07:36.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:07:36.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:36 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:07:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:36 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:07:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:36 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-142", "mode": "writeback"}]': finished 2024-03-21T13:07:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:36 smithi043 ceph-mon[30062]: osdmap e704: 8 total, 8 up, 8 in 2024-03-21T13:07:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:07:36.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:36 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:07:36.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:36 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:07:36.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:36 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-142", "mode": "writeback"}]': finished 2024-03-21T13:07:36.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:36 smithi043 ceph-mon[41699]: osdmap e704: 8 total, 8 up, 8 in 2024-03-21T13:07:36.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:07:36.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:36 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:07:37.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:37 smithi155 ceph-mon[33846]: pgmap v1095: 332 pgs: 332 active+clean; 4.4 MiB data, 719 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-21T13:07:37.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:37 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:07:37.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:37 smithi155 ceph-mon[33846]: osdmap e705: 8 total, 8 up, 8 in 2024-03-21T13:07:37.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:37 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:07:37.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:37 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:07:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:37 smithi043 ceph-mon[30062]: pgmap v1095: 332 pgs: 332 active+clean; 4.4 MiB data, 719 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-21T13:07:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:37 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:07:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:37 smithi043 ceph-mon[30062]: osdmap e705: 8 total, 8 up, 8 in 2024-03-21T13:07:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:37 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:07:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:37 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:07:37.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:37 smithi043 ceph-mon[41699]: pgmap v1095: 332 pgs: 332 active+clean; 4.4 MiB data, 719 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-21T13:07:37.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:37 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:07:37.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:37 smithi043 ceph-mon[41699]: osdmap e705: 8 total, 8 up, 8 in 2024-03-21T13:07:37.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:37 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:07:37.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:37 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:07:38.286 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:07:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:07:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:07:38.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:38 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:07:38.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:38 smithi155 ceph-mon[33846]: osdmap e706: 8 total, 8 up, 8 in 2024-03-21T13:07:38.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:38 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:07:38.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:38 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:07:38.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:38 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:07:38.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:38 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:07:38.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:38 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:07:38.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:38 smithi155 ceph-mon[33846]: osdmap e707: 8 total, 8 up, 8 in 2024-03-21T13:07:38.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:38 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:07:38.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:38 smithi043 ceph-mon[30062]: osdmap e706: 8 total, 8 up, 8 in 2024-03-21T13:07:38.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:38 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:07:38.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:38 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:07:38.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:38 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:07:38.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:38 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:07:38.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:38 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:07:38.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:38 smithi043 ceph-mon[30062]: osdmap e707: 8 total, 8 up, 8 in 2024-03-21T13:07:38.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:38 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:07:38.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:38 smithi043 ceph-mon[41699]: osdmap e706: 8 total, 8 up, 8 in 2024-03-21T13:07:38.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:38 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:07:38.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:38 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:07:38.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:38 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:07:38.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:38 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:07:38.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:38 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:07:38.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:38 smithi043 ceph-mon[41699]: osdmap e707: 8 total, 8 up, 8 in 2024-03-21T13:07:39.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:07:38 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:07:39.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:39 smithi155 ceph-mon[33846]: pgmap v1098: 332 pgs: 332 active+clean; 4.4 MiB data, 719 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:07:39.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:39 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:07:39.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:39 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:07:39.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:39 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-21T13:07:39.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:39 smithi155 ceph-mon[33846]: osdmap e708: 8 total, 8 up, 8 in 2024-03-21T13:07:39.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:39 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-21T13:07:39.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:39 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-21T13:07:39.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:39 smithi043 ceph-mon[30062]: pgmap v1098: 332 pgs: 332 active+clean; 4.4 MiB data, 719 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:07:39.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:39 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:07:39.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:39 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:07:39.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:39 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-21T13:07:39.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:39 smithi043 ceph-mon[30062]: osdmap e708: 8 total, 8 up, 8 in 2024-03-21T13:07:39.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:39 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-21T13:07:39.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:39 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-21T13:07:39.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:39 smithi043 ceph-mon[41699]: pgmap v1098: 332 pgs: 332 active+clean; 4.4 MiB data, 719 MiB used, 715 GiB / 715 GiB avail 2024-03-21T13:07:39.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:39 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:07:39.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:39 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:07:39.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:39 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-21T13:07:39.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:39 smithi043 ceph-mon[41699]: osdmap e708: 8 total, 8 up, 8 in 2024-03-21T13:07:39.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:39 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-21T13:07:39.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:39 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-21T13:07:40.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:40 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:07:40.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:40 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-21T13:07:40.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:40 smithi155 ceph-mon[33846]: osdmap e709: 8 total, 8 up, 8 in 2024-03-21T13:07:40.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:40 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-21T13:07:40.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:40 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-21T13:07:40.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:40 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:07:40.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:40 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-21T13:07:40.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:40 smithi043 ceph-mon[30062]: osdmap e709: 8 total, 8 up, 8 in 2024-03-21T13:07:40.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:40 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-21T13:07:40.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:40 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-21T13:07:40.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:40 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:07:40.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:40 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-21T13:07:40.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:40 smithi043 ceph-mon[41699]: osdmap e709: 8 total, 8 up, 8 in 2024-03-21T13:07:40.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:40 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-21T13:07:40.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:40 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-21T13:07:41.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:41 smithi155 ceph-mon[33846]: pgmap v1101: 332 pgs: 332 active+clean; 4.4 MiB data, 720 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:07:41.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:41 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-21T13:07:41.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:41 smithi155 ceph-mon[33846]: osdmap e710: 8 total, 8 up, 8 in 2024-03-21T13:07:41.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:41 smithi043 ceph-mon[30062]: pgmap v1101: 332 pgs: 332 active+clean; 4.4 MiB data, 720 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:07:41.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:41 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-21T13:07:41.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:41 smithi043 ceph-mon[30062]: osdmap e710: 8 total, 8 up, 8 in 2024-03-21T13:07:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:41 smithi043 ceph-mon[41699]: pgmap v1101: 332 pgs: 332 active+clean; 4.4 MiB data, 720 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:07:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:41 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-21T13:07:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:41 smithi043 ceph-mon[41699]: osdmap e710: 8 total, 8 up, 8 in 2024-03-21T13:07:42.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:07:42.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:42 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:07:42.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:07:42.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:42 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:07:42.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:07:42.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:42 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:07:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:43 smithi155 ceph-mon[33846]: pgmap v1104: 332 pgs: 332 active+clean; 4.4 MiB data, 720 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:07:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:43 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:07:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:43 smithi155 ceph-mon[33846]: osdmap e711: 8 total, 8 up, 8 in 2024-03-21T13:07:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-142"}]: dispatch 2024-03-21T13:07:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:43 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-142"}]: dispatch 2024-03-21T13:07:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:43 smithi043 ceph-mon[30062]: pgmap v1104: 332 pgs: 332 active+clean; 4.4 MiB data, 720 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:07:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:43 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:07:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:43 smithi043 ceph-mon[30062]: osdmap e711: 8 total, 8 up, 8 in 2024-03-21T13:07:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-142"}]: dispatch 2024-03-21T13:07:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:43 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-142"}]: dispatch 2024-03-21T13:07:43.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:43 smithi043 ceph-mon[41699]: pgmap v1104: 332 pgs: 332 active+clean; 4.4 MiB data, 720 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:07:43.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:43 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:07:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:43 smithi043 ceph-mon[41699]: osdmap e711: 8 total, 8 up, 8 in 2024-03-21T13:07:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-142"}]: dispatch 2024-03-21T13:07:43.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:43 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-142"}]: dispatch 2024-03-21T13:07:44.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:44 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-142"}]': finished 2024-03-21T13:07:44.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:44 smithi155 ceph-mon[33846]: osdmap e712: 8 total, 8 up, 8 in 2024-03-21T13:07:44.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:44 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:07:44.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:44 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:07:44.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:44 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-142"}]: dispatch 2024-03-21T13:07:44.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:44 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-142"}]: dispatch 2024-03-21T13:07:44.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:44 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-142"}]': finished 2024-03-21T13:07:44.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:44 smithi043 ceph-mon[30062]: osdmap e712: 8 total, 8 up, 8 in 2024-03-21T13:07:44.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:44 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:07:44.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:44 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:07:44.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:44 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-142"}]: dispatch 2024-03-21T13:07:44.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:44 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-142"}]: dispatch 2024-03-21T13:07:44.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:44 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-142"}]': finished 2024-03-21T13:07:44.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:44 smithi043 ceph-mon[41699]: osdmap e712: 8 total, 8 up, 8 in 2024-03-21T13:07:44.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:44 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:07:44.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:44 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:07:44.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:44 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-142"}]: dispatch 2024-03-21T13:07:44.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:44 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-142"}]: dispatch 2024-03-21T13:07:45.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:45 smithi155 ceph-mon[33846]: pgmap v1107: 332 pgs: 332 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:07:45.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:45 smithi155 ceph-mon[33846]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:07:45.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:45 smithi155 ceph-mon[33846]: osdmap e713: 8 total, 8 up, 8 in 2024-03-21T13:07:45.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:45 smithi043 ceph-mon[30062]: pgmap v1107: 332 pgs: 332 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:07:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:45 smithi043 ceph-mon[30062]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:07:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:45 smithi043 ceph-mon[30062]: osdmap e713: 8 total, 8 up, 8 in 2024-03-21T13:07:45.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:45 smithi043 ceph-mon[41699]: pgmap v1107: 332 pgs: 332 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:07:45.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:45 smithi043 ceph-mon[41699]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:07:45.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:45 smithi043 ceph-mon[41699]: osdmap e713: 8 total, 8 up, 8 in 2024-03-21T13:07:46.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:46 smithi155 ceph-mon[33846]: osdmap e714: 8 total, 8 up, 8 in 2024-03-21T13:07:46.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:07:46.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:46 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:07:46.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:46 smithi043 ceph-mon[30062]: osdmap e714: 8 total, 8 up, 8 in 2024-03-21T13:07:46.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:07:46.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:46 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:07:46.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:46 smithi043 ceph-mon[41699]: osdmap e714: 8 total, 8 up, 8 in 2024-03-21T13:07:46.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:07:46.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:46 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:07:47.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:47 smithi155 ceph-mon[33846]: pgmap v1110: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:07:47.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:47 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:07:47.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:47 smithi155 ceph-mon[33846]: osdmap e715: 8 total, 8 up, 8 in 2024-03-21T13:07:47.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:47 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:07:47.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:47 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:07:47.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:47 smithi043 ceph-mon[30062]: pgmap v1110: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:07:47.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:47 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:07:47.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:47 smithi043 ceph-mon[30062]: osdmap e715: 8 total, 8 up, 8 in 2024-03-21T13:07:47.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:47 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:07:47.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:47 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:07:47.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:47 smithi043 ceph-mon[41699]: pgmap v1110: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:07:47.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:47 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:07:47.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:47 smithi043 ceph-mon[41699]: osdmap e715: 8 total, 8 up, 8 in 2024-03-21T13:07:47.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:47 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:07:47.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:47 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:07:48.343 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:07:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:07:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:07:48.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:48 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-144", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:07:48.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:48 smithi155 ceph-mon[33846]: osdmap e716: 8 total, 8 up, 8 in 2024-03-21T13:07:48.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:48 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-144"}]: dispatch 2024-03-21T13:07:48.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:48 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-144"}]: dispatch 2024-03-21T13:07:48.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:48 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-144", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:07:48.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:48 smithi043 ceph-mon[30062]: osdmap e716: 8 total, 8 up, 8 in 2024-03-21T13:07:48.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:48 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-144"}]: dispatch 2024-03-21T13:07:48.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:48 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-144"}]: dispatch 2024-03-21T13:07:48.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:48 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-144", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:07:48.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:48 smithi043 ceph-mon[41699]: osdmap e716: 8 total, 8 up, 8 in 2024-03-21T13:07:48.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:48 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-144"}]: dispatch 2024-03-21T13:07:48.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:48 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-144"}]: dispatch 2024-03-21T13:07:49.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:07:48 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:07:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:49 smithi155 ceph-mon[33846]: pgmap v1113: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:07:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:49 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-144"}]': finished 2024-03-21T13:07:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:49 smithi155 ceph-mon[33846]: osdmap e717: 8 total, 8 up, 8 in 2024-03-21T13:07:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:49 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-144", "mode": "readproxy"}]: dispatch 2024-03-21T13:07:49.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:49 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-144", "mode": "readproxy"}]: dispatch 2024-03-21T13:07:49.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:49 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:07:49.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:49 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-144", "mode": "readproxy"}]': finished 2024-03-21T13:07:49.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:49 smithi155 ceph-mon[33846]: osdmap e718: 8 total, 8 up, 8 in 2024-03-21T13:07:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:49 smithi043 ceph-mon[30062]: pgmap v1113: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:07:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:49 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-144"}]': finished 2024-03-21T13:07:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:49 smithi043 ceph-mon[30062]: osdmap e717: 8 total, 8 up, 8 in 2024-03-21T13:07:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:49 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-144", "mode": "readproxy"}]: dispatch 2024-03-21T13:07:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:49 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-144", "mode": "readproxy"}]: dispatch 2024-03-21T13:07:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:49 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:07:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:49 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-144", "mode": "readproxy"}]': finished 2024-03-21T13:07:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:49 smithi043 ceph-mon[30062]: osdmap e718: 8 total, 8 up, 8 in 2024-03-21T13:07:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:49 smithi043 ceph-mon[41699]: pgmap v1113: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 738 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:07:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:49 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-144"}]': finished 2024-03-21T13:07:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:49 smithi043 ceph-mon[41699]: osdmap e717: 8 total, 8 up, 8 in 2024-03-21T13:07:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:49 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-144", "mode": "readproxy"}]: dispatch 2024-03-21T13:07:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:49 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-144", "mode": "readproxy"}]: dispatch 2024-03-21T13:07:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:49 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:07:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:49 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-144", "mode": "readproxy"}]': finished 2024-03-21T13:07:49.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:49 smithi043 ceph-mon[41699]: osdmap e718: 8 total, 8 up, 8 in 2024-03-21T13:07:51.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:50 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:07:51.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:50 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:07:51.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:50 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:07:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:50 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:07:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:50 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:07:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:50 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:07:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:50 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:07:51.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:50 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:07:51.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:50 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:07:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:51 smithi155 ceph-mon[33846]: pgmap v1116: 332 pgs: 332 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:07:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:51 smithi043 ceph-mon[30062]: pgmap v1116: 332 pgs: 332 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:07:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:51 smithi043 ceph-mon[41699]: pgmap v1116: 332 pgs: 332 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:07:53.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:52 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:07:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:52 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:07:53.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:52 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:07:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:53 smithi155 ceph-mon[33846]: pgmap v1117: 332 pgs: 332 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 876 B/s rd, 175 B/s wr, 1 op/s 2024-03-21T13:07:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:53 smithi043 ceph-mon[30062]: pgmap v1117: 332 pgs: 332 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 876 B/s rd, 175 B/s wr, 1 op/s 2024-03-21T13:07:54.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:53 smithi043 ceph-mon[41699]: pgmap v1117: 332 pgs: 332 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 876 B/s rd, 175 B/s wr, 1 op/s 2024-03-21T13:07:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:55 smithi155 ceph-mon[33846]: pgmap v1118: 332 pgs: 332 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 149 B/s wr, 1 op/s 2024-03-21T13:07:56.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:55 smithi043 ceph-mon[30062]: pgmap v1118: 332 pgs: 332 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 149 B/s wr, 1 op/s 2024-03-21T13:07:56.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:55 smithi043 ceph-mon[41699]: pgmap v1118: 332 pgs: 332 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 149 B/s wr, 1 op/s 2024-03-21T13:07:58.023 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:57 smithi043 ceph-mon[30062]: pgmap v1119: 332 pgs: 332 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 127 B/s wr, 1 op/s 2024-03-21T13:07:58.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:57 smithi043 ceph-mon[41699]: pgmap v1119: 332 pgs: 332 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 127 B/s wr, 1 op/s 2024-03-21T13:07:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:57 smithi155 ceph-mon[33846]: pgmap v1119: 332 pgs: 332 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 127 B/s wr, 1 op/s 2024-03-21T13:07:58.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:07:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:07:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:07:59.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:07:58 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:08:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:59 smithi155 ceph-mon[33846]: pgmap v1120: 332 pgs: 332 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 624 B/s rd, 0 op/s 2024-03-21T13:08:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:59 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:07:59 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:59 smithi043 ceph-mon[30062]: pgmap v1120: 332 pgs: 332 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 624 B/s rd, 0 op/s 2024-03-21T13:08:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:59 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:07:59 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:59 smithi043 ceph-mon[41699]: pgmap v1120: 332 pgs: 332 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 624 B/s rd, 0 op/s 2024-03-21T13:08:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:59 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:07:59 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:00 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:08:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:00 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:08:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:00 smithi155 ceph-mon[33846]: osdmap e719: 8 total, 8 up, 8 in 2024-03-21T13:08:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:00 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-144"}]: dispatch 2024-03-21T13:08:01.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:00 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-144"}]: dispatch 2024-03-21T13:08:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:00 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:08:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:00 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:08:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:00 smithi043 ceph-mon[30062]: osdmap e719: 8 total, 8 up, 8 in 2024-03-21T13:08:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:00 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-144"}]: dispatch 2024-03-21T13:08:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:00 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-144"}]: dispatch 2024-03-21T13:08:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:00 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:08:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:00 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:08:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:00 smithi043 ceph-mon[41699]: osdmap e719: 8 total, 8 up, 8 in 2024-03-21T13:08:01.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:00 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-144"}]: dispatch 2024-03-21T13:08:01.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:00 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-144"}]: dispatch 2024-03-21T13:08:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:01 smithi155 ceph-mon[33846]: pgmap v1122: 332 pgs: 332 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-21T13:08:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:01 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:08:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:01 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-144"}]': finished 2024-03-21T13:08:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:01 smithi155 ceph-mon[33846]: osdmap e720: 8 total, 8 up, 8 in 2024-03-21T13:08:02.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:01 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:02.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:01 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:02.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:01 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-144"}]: dispatch 2024-03-21T13:08:02.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:01 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-144"}]: dispatch 2024-03-21T13:08:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:01 smithi043 ceph-mon[30062]: pgmap v1122: 332 pgs: 332 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-21T13:08:02.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:01 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:08:02.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:01 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-144"}]': finished 2024-03-21T13:08:02.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:01 smithi043 ceph-mon[30062]: osdmap e720: 8 total, 8 up, 8 in 2024-03-21T13:08:02.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:01 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:02.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:01 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:02.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:01 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-144"}]: dispatch 2024-03-21T13:08:02.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:01 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-144"}]: dispatch 2024-03-21T13:08:02.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:01 smithi043 ceph-mon[41699]: pgmap v1122: 332 pgs: 332 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-21T13:08:02.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:01 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:08:02.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:01 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-144"}]': finished 2024-03-21T13:08:02.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:01 smithi043 ceph-mon[41699]: osdmap e720: 8 total, 8 up, 8 in 2024-03-21T13:08:02.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:01 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:02.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:01 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:02.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:01 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-144"}]: dispatch 2024-03-21T13:08:02.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:01 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-144"}]: dispatch 2024-03-21T13:08:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:02 smithi155 ceph-mon[33846]: osdmap e721: 8 total, 8 up, 8 in 2024-03-21T13:08:03.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:02 smithi043 ceph-mon[30062]: osdmap e721: 8 total, 8 up, 8 in 2024-03-21T13:08:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:02 smithi043 ceph-mon[41699]: osdmap e721: 8 total, 8 up, 8 in 2024-03-21T13:08:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:03 smithi155 ceph-mon[33846]: pgmap v1125: 300 pgs: 300 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:08:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:03 smithi155 ceph-mon[33846]: osdmap e722: 8 total, 8 up, 8 in 2024-03-21T13:08:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:03 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:03 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:03 smithi043 ceph-mon[30062]: pgmap v1125: 300 pgs: 300 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:08:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:03 smithi043 ceph-mon[30062]: osdmap e722: 8 total, 8 up, 8 in 2024-03-21T13:08:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:03 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:03 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:03 smithi043 ceph-mon[41699]: pgmap v1125: 300 pgs: 300 active+clean; 4.4 MiB data, 756 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:08:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:03 smithi043 ceph-mon[41699]: osdmap e722: 8 total, 8 up, 8 in 2024-03-21T13:08:04.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:03 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:04.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:03 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:05.108 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:04 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:08:05.108 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:04 smithi155 ceph-mon[33846]: osdmap e723: 8 total, 8 up, 8 in 2024-03-21T13:08:05.108 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:08:05.108 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:04 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:08:05.108 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:04 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:08:05.108 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:04 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:08:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:04 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:08:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:04 smithi043 ceph-mon[30062]: osdmap e723: 8 total, 8 up, 8 in 2024-03-21T13:08:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:08:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:04 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:08:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:04 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:08:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:04 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:08:05.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:04 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:08:05.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:04 smithi043 ceph-mon[41699]: osdmap e723: 8 total, 8 up, 8 in 2024-03-21T13:08:05.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:08:05.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:04 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:08:05.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:04 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:08:05.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:04 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:08:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:05 smithi155 ceph-mon[33846]: pgmap v1128: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:08:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:05 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-146", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:08:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:05 smithi155 ceph-mon[33846]: osdmap e724: 8 total, 8 up, 8 in 2024-03-21T13:08:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:05 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-146"}]: dispatch 2024-03-21T13:08:06.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:05 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-146"}]: dispatch 2024-03-21T13:08:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:05 smithi043 ceph-mon[30062]: pgmap v1128: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:08:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:05 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-146", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:08:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:05 smithi043 ceph-mon[30062]: osdmap e724: 8 total, 8 up, 8 in 2024-03-21T13:08:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:05 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-146"}]: dispatch 2024-03-21T13:08:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:05 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-146"}]: dispatch 2024-03-21T13:08:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:05 smithi043 ceph-mon[41699]: pgmap v1128: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:08:06.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:05 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-146", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:08:06.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:05 smithi043 ceph-mon[41699]: osdmap e724: 8 total, 8 up, 8 in 2024-03-21T13:08:06.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:05 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-146"}]: dispatch 2024-03-21T13:08:06.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:05 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-146"}]: dispatch 2024-03-21T13:08:07.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:06 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-146"}]': finished 2024-03-21T13:08:07.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:06 smithi155 ceph-mon[33846]: osdmap e725: 8 total, 8 up, 8 in 2024-03-21T13:08:07.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:06 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-146", "mode": "writeback"}]: dispatch 2024-03-21T13:08:07.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:06 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-146", "mode": "writeback"}]: dispatch 2024-03-21T13:08:07.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:06 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:08:07.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:06 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-146", "mode": "writeback"}]': finished 2024-03-21T13:08:07.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:06 smithi155 ceph-mon[33846]: osdmap e726: 8 total, 8 up, 8 in 2024-03-21T13:08:07.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:06 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-146"}]': finished 2024-03-21T13:08:07.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:06 smithi043 ceph-mon[30062]: osdmap e725: 8 total, 8 up, 8 in 2024-03-21T13:08:07.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:06 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-146", "mode": "writeback"}]: dispatch 2024-03-21T13:08:07.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:06 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-146", "mode": "writeback"}]: dispatch 2024-03-21T13:08:07.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:06 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:08:07.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:06 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-146", "mode": "writeback"}]': finished 2024-03-21T13:08:07.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:06 smithi043 ceph-mon[30062]: osdmap e726: 8 total, 8 up, 8 in 2024-03-21T13:08:07.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:06 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi043-81880-111", "overlaypool": "test-rados-api-smithi043-81880-146"}]': finished 2024-03-21T13:08:07.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:06 smithi043 ceph-mon[41699]: osdmap e725: 8 total, 8 up, 8 in 2024-03-21T13:08:07.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:06 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-146", "mode": "writeback"}]: dispatch 2024-03-21T13:08:07.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:06 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-146", "mode": "writeback"}]: dispatch 2024-03-21T13:08:07.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:06 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-21T13:08:07.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:06 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi043-81880-146", "mode": "writeback"}]': finished 2024-03-21T13:08:07.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:06 smithi043 ceph-mon[41699]: osdmap e726: 8 total, 8 up, 8 in 2024-03-21T13:08:08.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:07 smithi043 ceph-mon[30062]: pgmap v1131: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:08:08.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:07 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:08:08.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:07 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:08:08.136 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:08:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:08:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:08:08.136 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:07 smithi043 ceph-mon[41699]: pgmap v1131: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:08:08.136 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:07 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:08:08.136 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:07 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:08:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:07 smithi155 ceph-mon[33846]: pgmap v1131: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:08:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:07 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:08:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:07 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_count","val": "2"}]: dispatch 2024-03-21T13:08:09.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:08:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:08:09.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:08 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:08:09.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:08 smithi155 ceph-mon[33846]: osdmap e727: 8 total, 8 up, 8 in 2024-03-21T13:08:09.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:08 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:08:09.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:08 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:08:09.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:08 smithi155 ceph-mon[33846]: pgmap v1134: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:08:09.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:08 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:08:09.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:08 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:08:09.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:08 smithi043 ceph-mon[30062]: osdmap e727: 8 total, 8 up, 8 in 2024-03-21T13:08:09.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:08 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:08:09.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:08 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:08:09.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:08 smithi043 ceph-mon[30062]: pgmap v1134: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:08:09.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:08 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:08:09.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:08 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_count","val": "2"}]': finished 2024-03-21T13:08:09.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:08 smithi043 ceph-mon[41699]: osdmap e727: 8 total, 8 up, 8 in 2024-03-21T13:08:09.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:08 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:08:09.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:08 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_period","val": "600"}]: dispatch 2024-03-21T13:08:09.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:08 smithi043 ceph-mon[41699]: pgmap v1134: 332 pgs: 32 creating+peering, 300 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:08:09.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:08 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:08:10.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:09 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:08:10.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:09 smithi155 ceph-mon[33846]: osdmap e728: 8 total, 8 up, 8 in 2024-03-21T13:08:10.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:09 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:08:10.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:09 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:08:10.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:09 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:08:10.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:09 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:08:10.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:09 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:08:10.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:09 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:08:10.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:09 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:08:10.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:09 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:08:10.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:09 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:08:10.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:09 smithi043 ceph-mon[30062]: osdmap e728: 8 total, 8 up, 8 in 2024-03-21T13:08:10.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:09 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:08:10.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:09 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:08:10.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:09 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:08:10.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:09 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:08:10.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:09 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:08:10.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:09 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:08:10.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:09 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:08:10.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:09 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:08:10.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:09 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_period","val": "600"}]': finished 2024-03-21T13:08:10.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:09 smithi043 ceph-mon[41699]: osdmap e728: 8 total, 8 up, 8 in 2024-03-21T13:08:10.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:09 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:08:10.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:09 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-21T13:08:10.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:09 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:08:10.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:09 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:08:10.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:09 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:08:10.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:09 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:08:10.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:09 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:08:10.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:09 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:08:11.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:10 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:08:11.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:10 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:08:11.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:10 smithi155 ceph-mon[33846]: osdmap e729: 8 total, 8 up, 8 in 2024-03-21T13:08:11.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:08:11.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:10 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:08:11.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:10 smithi155 ceph-mon[33846]: pgmap v1137: 332 pgs: 332 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-21T13:08:11.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:10 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:08:11.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:10 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:08:11.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:10 smithi043 ceph-mon[30062]: osdmap e729: 8 total, 8 up, 8 in 2024-03-21T13:08:11.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:08:11.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:10 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:08:11.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:10 smithi043 ceph-mon[30062]: pgmap v1137: 332 pgs: 332 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-21T13:08:11.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:10 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-21T13:08:11.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:10 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "hit_set_type","val": "bloom"}]': finished 2024-03-21T13:08:11.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:10 smithi043 ceph-mon[41699]: osdmap e729: 8 total, 8 up, 8 in 2024-03-21T13:08:11.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:08:11.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:10 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-21T13:08:11.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:10 smithi043 ceph-mon[41699]: pgmap v1137: 332 pgs: 332 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-21T13:08:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:11 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-21T13:08:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:11 smithi155 ceph-mon[33846]: osdmap e730: 8 total, 8 up, 8 in 2024-03-21T13:08:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:11 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "target_max_objects","val": "1"}]: dispatch 2024-03-21T13:08:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:11 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "target_max_objects","val": "1"}]: dispatch 2024-03-21T13:08:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:11 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "target_max_objects","val": "1"}]': finished 2024-03-21T13:08:12.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:11 smithi155 ceph-mon[33846]: osdmap e731: 8 total, 8 up, 8 in 2024-03-21T13:08:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:11 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-21T13:08:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:11 smithi043 ceph-mon[30062]: osdmap e730: 8 total, 8 up, 8 in 2024-03-21T13:08:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:11 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "target_max_objects","val": "1"}]: dispatch 2024-03-21T13:08:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:11 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "target_max_objects","val": "1"}]: dispatch 2024-03-21T13:08:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:11 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "target_max_objects","val": "1"}]': finished 2024-03-21T13:08:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:11 smithi043 ceph-mon[30062]: osdmap e731: 8 total, 8 up, 8 in 2024-03-21T13:08:12.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:11 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-21T13:08:12.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:11 smithi043 ceph-mon[41699]: osdmap e730: 8 total, 8 up, 8 in 2024-03-21T13:08:12.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:11 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "target_max_objects","val": "1"}]: dispatch 2024-03-21T13:08:12.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:11 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "target_max_objects","val": "1"}]: dispatch 2024-03-21T13:08:12.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:11 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-146","var": "target_max_objects","val": "1"}]': finished 2024-03-21T13:08:12.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:11 smithi043 ceph-mon[41699]: osdmap e731: 8 total, 8 up, 8 in 2024-03-21T13:08:13.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:12 smithi155 ceph-mon[33846]: pgmap v1140: 332 pgs: 332 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-21T13:08:13.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:12 smithi043 ceph-mon[30062]: pgmap v1140: 332 pgs: 332 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-21T13:08:13.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:12 smithi043 ceph-mon[41699]: pgmap v1140: 332 pgs: 332 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-03-21T13:08:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:13 smithi155 ceph-mon[33846]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-21T13:08:14.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:13 smithi043 ceph-mon[30062]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-21T13:08:14.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:13 smithi043 ceph-mon[41699]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-21T13:08:15.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:14 smithi155 ceph-mon[33846]: pgmap v1141: 332 pgs: 332 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 957 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:08:15.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:14 smithi043 ceph-mon[30062]: pgmap v1141: 332 pgs: 332 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 957 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:08:15.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:14 smithi043 ceph-mon[41699]: pgmap v1141: 332 pgs: 332 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 957 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:08:16.957 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 13:08:16 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=sqlstore.transactions t=2024-03-21T13:08:16.529191062Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-03-21T13:08:17.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:17 smithi155 ceph-mon[33846]: pgmap v1142: 332 pgs: 332 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 806 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:08:17.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:17 smithi043 ceph-mon[30062]: pgmap v1142: 332 pgs: 332 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 806 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:08:17.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:17 smithi043 ceph-mon[41699]: pgmap v1142: 332 pgs: 332 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 806 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:08:18.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:08:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:08:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:08:19.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:08:18 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:08:19.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:19 smithi155 ceph-mon[33846]: pgmap v1143: 332 pgs: 332 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-03-21T13:08:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:19 smithi043 ceph-mon[30062]: pgmap v1143: 332 pgs: 332 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-03-21T13:08:19.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:19 smithi043 ceph-mon[41699]: pgmap v1143: 332 pgs: 332 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-03-21T13:08:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:20 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:08:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:20 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:08:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:20 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:08:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:20 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:08:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:20 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:08:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:20 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:08:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:20 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:08:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:20 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:08:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:20 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:08:22.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:21 smithi155 ceph-mon[33846]: pgmap v1144: 332 pgs: 332 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:08:22.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:21 smithi043 ceph-mon[30062]: pgmap v1144: 332 pgs: 332 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:08:22.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:21 smithi043 ceph-mon[41699]: pgmap v1144: 332 pgs: 332 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:08:23.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:22 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:23.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:22 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:22 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:23.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:22 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:23.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:22 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:23.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:22 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:24.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:23 smithi155 ceph-mon[33846]: pgmap v1145: 332 pgs: 332 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 989 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:08:24.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:23 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:08:24.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:23 smithi155 ceph-mon[33846]: osdmap e732: 8 total, 8 up, 8 in 2024-03-21T13:08:24.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:23 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-146"}]: dispatch 2024-03-21T13:08:24.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:23 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-146"}]: dispatch 2024-03-21T13:08:24.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:23 smithi043 ceph-mon[41699]: pgmap v1145: 332 pgs: 332 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 989 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:08:24.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:23 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:08:24.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:23 smithi043 ceph-mon[41699]: osdmap e732: 8 total, 8 up, 8 in 2024-03-21T13:08:24.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:23 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-146"}]: dispatch 2024-03-21T13:08:24.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:23 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-146"}]: dispatch 2024-03-21T13:08:24.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:23 smithi043 ceph-mon[30062]: pgmap v1145: 332 pgs: 332 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 989 B/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:08:24.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:23 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:08:24.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:23 smithi043 ceph-mon[30062]: osdmap e732: 8 total, 8 up, 8 in 2024-03-21T13:08:24.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:23 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-146"}]: dispatch 2024-03-21T13:08:24.236 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:23 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-146"}]: dispatch 2024-03-21T13:08:25.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:24 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-146"}]': finished 2024-03-21T13:08:25.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:24 smithi155 ceph-mon[33846]: osdmap e733: 8 total, 8 up, 8 in 2024-03-21T13:08:25.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:24 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:25.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:24 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:25.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:24 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-146"}]: dispatch 2024-03-21T13:08:25.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:24 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-146"}]: dispatch 2024-03-21T13:08:25.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:24 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-146"}]': finished 2024-03-21T13:08:25.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:24 smithi043 ceph-mon[30062]: osdmap e733: 8 total, 8 up, 8 in 2024-03-21T13:08:25.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:24 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:25.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:24 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:25.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:24 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-146"}]: dispatch 2024-03-21T13:08:25.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:24 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-146"}]: dispatch 2024-03-21T13:08:25.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:24 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-146"}]': finished 2024-03-21T13:08:25.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:24 smithi043 ceph-mon[41699]: osdmap e733: 8 total, 8 up, 8 in 2024-03-21T13:08:25.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:24 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:25.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:24 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:25.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:24 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-146"}]: dispatch 2024-03-21T13:08:25.237 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:24 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-146"}]: dispatch 2024-03-21T13:08:26.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:25 smithi155 ceph-mon[33846]: pgmap v1148: 332 pgs: 332 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-21T13:08:26.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:25 smithi155 ceph-mon[33846]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:08:26.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:25 smithi155 ceph-mon[33846]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-21T13:08:26.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:25 smithi155 ceph-mon[33846]: osdmap e734: 8 total, 8 up, 8 in 2024-03-21T13:08:26.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:25 smithi043 ceph-mon[30062]: pgmap v1148: 332 pgs: 332 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-21T13:08:26.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:25 smithi043 ceph-mon[30062]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:08:26.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:25 smithi043 ceph-mon[30062]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-21T13:08:26.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:25 smithi043 ceph-mon[30062]: osdmap e734: 8 total, 8 up, 8 in 2024-03-21T13:08:26.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:25 smithi043 ceph-mon[41699]: pgmap v1148: 332 pgs: 332 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-21T13:08:26.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:25 smithi043 ceph-mon[41699]: Health check update: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:08:26.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:25 smithi043 ceph-mon[41699]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-21T13:08:26.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:25 smithi043 ceph-mon[41699]: osdmap e734: 8 total, 8 up, 8 in 2024-03-21T13:08:27.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:26 smithi155 ceph-mon[33846]: osdmap e735: 8 total, 8 up, 8 in 2024-03-21T13:08:27.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:26 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:27.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:26 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:27.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:26 smithi155 ceph-mon[33846]: pgmap v1151: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-21T13:08:27.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:26 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:08:27.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:26 smithi155 ceph-mon[33846]: osdmap e736: 8 total, 8 up, 8 in 2024-03-21T13:08:27.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:26 smithi043 ceph-mon[30062]: osdmap e735: 8 total, 8 up, 8 in 2024-03-21T13:08:27.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:26 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:27.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:26 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:27.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:26 smithi043 ceph-mon[30062]: pgmap v1151: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-21T13:08:27.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:26 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:08:27.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:26 smithi043 ceph-mon[30062]: osdmap e736: 8 total, 8 up, 8 in 2024-03-21T13:08:27.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:26 smithi043 ceph-mon[41699]: osdmap e735: 8 total, 8 up, 8 in 2024-03-21T13:08:27.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:26 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:27.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:26 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:27.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:26 smithi043 ceph-mon[41699]: pgmap v1151: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-21T13:08:27.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:26 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:08:27.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:26 smithi043 ceph-mon[41699]: osdmap e736: 8 total, 8 up, 8 in 2024-03-21T13:08:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:27 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:08:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:27 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:08:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:27 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-148", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:08:28.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:27 smithi155 ceph-mon[33846]: osdmap e737: 8 total, 8 up, 8 in 2024-03-21T13:08:28.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:08:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:08:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:08:28.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:27 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:08:28.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:27 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:08:28.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:27 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-148", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:08:28.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:27 smithi043 ceph-mon[30062]: osdmap e737: 8 total, 8 up, 8 in 2024-03-21T13:08:28.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:27 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:08:28.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:27 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-21T13:08:28.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:27 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-148", "force_nonempty": "--force-nonempty" }]': finished 2024-03-21T13:08:28.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:27 smithi043 ceph-mon[41699]: osdmap e737: 8 total, 8 up, 8 in 2024-03-21T13:08:29.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:08:28 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:08:29.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-148"}]: dispatch 2024-03-21T13:08:29.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:28 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-148"}]: dispatch 2024-03-21T13:08:29.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:28 smithi155 ceph-mon[33846]: pgmap v1154: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:08:29.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-148"}]: dispatch 2024-03-21T13:08:29.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:28 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-148"}]: dispatch 2024-03-21T13:08:29.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:28 smithi043 ceph-mon[30062]: pgmap v1154: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:08:29.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-148"}]: dispatch 2024-03-21T13:08:29.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:28 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-148"}]: dispatch 2024-03-21T13:08:29.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:28 smithi043 ceph-mon[41699]: pgmap v1154: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:08:30.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:29 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-148"}]': finished 2024-03-21T13:08:30.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:29 smithi155 ceph-mon[33846]: osdmap e738: 8 total, 8 up, 8 in 2024-03-21T13:08:30.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:29 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:30.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:29 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:30.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:29 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-148"}]: dispatch 2024-03-21T13:08:30.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:29 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-148"}]: dispatch 2024-03-21T13:08:30.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:29 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:08:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:29 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-148"}]': finished 2024-03-21T13:08:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:29 smithi043 ceph-mon[30062]: osdmap e738: 8 total, 8 up, 8 in 2024-03-21T13:08:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:29 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:29 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:29 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-148"}]: dispatch 2024-03-21T13:08:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:29 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-148"}]: dispatch 2024-03-21T13:08:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:29 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:08:30.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:29 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-148"}]': finished 2024-03-21T13:08:30.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:29 smithi043 ceph-mon[41699]: osdmap e738: 8 total, 8 up, 8 in 2024-03-21T13:08:30.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:29 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:30.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:29 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:30.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:29 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-148"}]: dispatch 2024-03-21T13:08:30.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:29 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-148"}]: dispatch 2024-03-21T13:08:30.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:29 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:08:31.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:30 smithi155 ceph-mon[33846]: osdmap e739: 8 total, 8 up, 8 in 2024-03-21T13:08:31.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:30 smithi155 ceph-mon[33846]: pgmap v1157: 300 pgs: 300 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:08:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:30 smithi043 ceph-mon[30062]: osdmap e739: 8 total, 8 up, 8 in 2024-03-21T13:08:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:30 smithi043 ceph-mon[30062]: pgmap v1157: 300 pgs: 300 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:08:31.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:30 smithi043 ceph-mon[41699]: osdmap e739: 8 total, 8 up, 8 in 2024-03-21T13:08:31.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:30 smithi043 ceph-mon[41699]: pgmap v1157: 300 pgs: 300 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:08:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:31 smithi155 ceph-mon[33846]: osdmap e740: 8 total, 8 up, 8 in 2024-03-21T13:08:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:31 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:31 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:31 smithi043 ceph-mon[41699]: osdmap e740: 8 total, 8 up, 8 in 2024-03-21T13:08:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:31 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:31 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:31 smithi043 ceph-mon[30062]: osdmap e740: 8 total, 8 up, 8 in 2024-03-21T13:08:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:31 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:31 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:33.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:32 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:08:33.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:32 smithi155 ceph-mon[33846]: osdmap e741: 8 total, 8 up, 8 in 2024-03-21T13:08:33.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:32 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:33.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:32 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:33.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:32 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-150"}]: dispatch 2024-03-21T13:08:33.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:32 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-150"}]: dispatch 2024-03-21T13:08:33.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:32 smithi155 ceph-mon[33846]: pgmap v1160: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:08:33.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:32 smithi155 ceph-mon[33846]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:08:33.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:32 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:08:33.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:32 smithi043 ceph-mon[30062]: osdmap e741: 8 total, 8 up, 8 in 2024-03-21T13:08:33.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:32 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:33.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:32 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:33.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:32 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-150"}]: dispatch 2024-03-21T13:08:33.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:32 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-150"}]: dispatch 2024-03-21T13:08:33.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:32 smithi043 ceph-mon[30062]: pgmap v1160: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:08:33.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:32 smithi043 ceph-mon[30062]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:08:33.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:32 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:08:33.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:32 smithi043 ceph-mon[41699]: osdmap e741: 8 total, 8 up, 8 in 2024-03-21T13:08:33.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:32 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:33.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:32 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:33.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:32 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-150"}]: dispatch 2024-03-21T13:08:33.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:32 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-150"}]: dispatch 2024-03-21T13:08:33.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:32 smithi043 ceph-mon[41699]: pgmap v1160: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-21T13:08:33.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:32 smithi043 ceph-mon[41699]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:08:34.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:33 smithi155 ceph-mon[33846]: osdmap e742: 8 total, 8 up, 8 in 2024-03-21T13:08:34.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:33 smithi043 ceph-mon[30062]: osdmap e742: 8 total, 8 up, 8 in 2024-03-21T13:08:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:33 smithi043 ceph-mon[41699]: osdmap e742: 8 total, 8 up, 8 in 2024-03-21T13:08:35.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:34 smithi155 ceph-mon[33846]: osdmap e743: 8 total, 8 up, 8 in 2024-03-21T13:08:35.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:34 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:35.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:34 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:35.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:34 smithi155 ceph-mon[33846]: pgmap v1163: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 764 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:08:35.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:34 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:08:35.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:34 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:08:35.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:34 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:08:35.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:34 smithi155 ceph-mon[33846]: osdmap e744: 8 total, 8 up, 8 in 2024-03-21T13:08:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:34 smithi043 ceph-mon[30062]: osdmap e743: 8 total, 8 up, 8 in 2024-03-21T13:08:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:34 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:34 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:34 smithi043 ceph-mon[30062]: pgmap v1163: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 764 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:08:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:34 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:08:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:34 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:08:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:34 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:08:35.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:34 smithi043 ceph-mon[30062]: osdmap e744: 8 total, 8 up, 8 in 2024-03-21T13:08:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:34 smithi043 ceph-mon[41699]: osdmap e743: 8 total, 8 up, 8 in 2024-03-21T13:08:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:34 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:34 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:34 smithi043 ceph-mon[41699]: pgmap v1163: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 764 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:08:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:34 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:08:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:34 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:08:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:34 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:08:35.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:34 smithi043 ceph-mon[41699]: osdmap e744: 8 total, 8 up, 8 in 2024-03-21T13:08:36.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:36.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:36 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:36.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:36 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-152"}]: dispatch 2024-03-21T13:08:36.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:36 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-152"}]: dispatch 2024-03-21T13:08:36.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:36.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:36 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:36.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:36 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-152"}]: dispatch 2024-03-21T13:08:36.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:36 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-152"}]: dispatch 2024-03-21T13:08:36.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:36.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:36 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:36.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:36 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-152"}]: dispatch 2024-03-21T13:08:36.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:36 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-152"}]: dispatch 2024-03-21T13:08:37.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:37 smithi155 ceph-mon[33846]: osdmap e745: 8 total, 8 up, 8 in 2024-03-21T13:08:37.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:37 smithi155 ceph-mon[33846]: pgmap v1166: 300 pgs: 300 active+clean; 4.4 MiB data, 764 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:08:37.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:37 smithi155 ceph-mon[33846]: osdmap e746: 8 total, 8 up, 8 in 2024-03-21T13:08:37.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:37 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:37.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:37 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:37.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:37 smithi043 ceph-mon[30062]: osdmap e745: 8 total, 8 up, 8 in 2024-03-21T13:08:37.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:37 smithi043 ceph-mon[30062]: pgmap v1166: 300 pgs: 300 active+clean; 4.4 MiB data, 764 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:08:37.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:37 smithi043 ceph-mon[30062]: osdmap e746: 8 total, 8 up, 8 in 2024-03-21T13:08:37.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:37 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:37.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:37 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:37.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:37 smithi043 ceph-mon[41699]: osdmap e745: 8 total, 8 up, 8 in 2024-03-21T13:08:37.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:37 smithi043 ceph-mon[41699]: pgmap v1166: 300 pgs: 300 active+clean; 4.4 MiB data, 764 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-21T13:08:37.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:37 smithi043 ceph-mon[41699]: osdmap e746: 8 total, 8 up, 8 in 2024-03-21T13:08:37.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:37 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:37.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:37 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:08:38.385 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:08:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:08:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:08:39.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:08:38 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:08:39.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:38 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:08:39.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:38 smithi155 ceph-mon[33846]: osdmap e747: 8 total, 8 up, 8 in 2024-03-21T13:08:39.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:38 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-111","var": "dedup_tier","val": "test-rados-api-smithi043-81880-154"}]: dispatch 2024-03-21T13:08:39.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:38 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-111","var": "dedup_tier","val": "test-rados-api-smithi043-81880-154"}]: dispatch 2024-03-21T13:08:39.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:38 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:39.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:38 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:39.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:38 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-154"}]: dispatch 2024-03-21T13:08:39.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:38 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-154"}]: dispatch 2024-03-21T13:08:39.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:38 smithi155 ceph-mon[33846]: pgmap v1169: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 764 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:08:39.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:38 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:08:39.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:38 smithi043 ceph-mon[30062]: osdmap e747: 8 total, 8 up, 8 in 2024-03-21T13:08:39.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:38 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-111","var": "dedup_tier","val": "test-rados-api-smithi043-81880-154"}]: dispatch 2024-03-21T13:08:39.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:38 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-111","var": "dedup_tier","val": "test-rados-api-smithi043-81880-154"}]: dispatch 2024-03-21T13:08:39.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:38 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:39.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:38 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:39.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:38 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-154"}]: dispatch 2024-03-21T13:08:39.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:38 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-154"}]: dispatch 2024-03-21T13:08:39.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:38 smithi043 ceph-mon[30062]: pgmap v1169: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 764 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:08:39.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:38 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi043-81880-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:08:39.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:38 smithi043 ceph-mon[41699]: osdmap e747: 8 total, 8 up, 8 in 2024-03-21T13:08:39.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:38 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-111","var": "dedup_tier","val": "test-rados-api-smithi043-81880-154"}]: dispatch 2024-03-21T13:08:39.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:38 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi043-81880-111","var": "dedup_tier","val": "test-rados-api-smithi043-81880-154"}]: dispatch 2024-03-21T13:08:39.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:38 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:39.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:38 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:39.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:38 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-154"}]: dispatch 2024-03-21T13:08:39.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:38 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi043-81880-111", "tierpool": "test-rados-api-smithi043-81880-154"}]: dispatch 2024-03-21T13:08:39.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:38 smithi043 ceph-mon[41699]: pgmap v1169: 332 pgs: 32 unknown, 300 active+clean; 4.4 MiB data, 764 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:08:40.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:39 smithi155 ceph-mon[33846]: osdmap e748: 8 total, 8 up, 8 in 2024-03-21T13:08:40.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:39 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:08:40.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:39 smithi043 ceph-mon[30062]: osdmap e748: 8 total, 8 up, 8 in 2024-03-21T13:08:40.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:39 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:08:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:39 smithi043 ceph-mon[41699]: osdmap e748: 8 total, 8 up, 8 in 2024-03-21T13:08:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:39 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:08:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:40 smithi155 ceph-mon[33846]: osdmap e749: 8 total, 8 up, 8 in 2024-03-21T13:08:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:40 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:40 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:40 smithi155 ceph-mon[33846]: pgmap v1172: 292 pgs: 292 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:08:41.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:40 smithi043 ceph-mon[30062]: osdmap e749: 8 total, 8 up, 8 in 2024-03-21T13:08:41.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:40 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:41.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:40 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:41.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:40 smithi043 ceph-mon[30062]: pgmap v1172: 292 pgs: 292 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:08:41.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:40 smithi043 ceph-mon[41699]: osdmap e749: 8 total, 8 up, 8 in 2024-03-21T13:08:41.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:40 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:41.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:40 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:41.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:40 smithi043 ceph-mon[41699]: pgmap v1172: 292 pgs: 292 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:08:41.945 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting LibRadosTwoPoolsECPP.TryFlush (7898 ms) 2024-03-21T13:08:41.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-03-21T13:08:41.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (13049 ms) 2024-03-21T13:08:41.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-03-21T13:08:41.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8065 ms) 2024-03-21T13:08:41.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-03-21T13:08:41.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13114 ms) 2024-03-21T13:08:41.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-03-21T13:08:41.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7795 ms) 2024-03-21T13:08:41.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-03-21T13:08:41.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8046 ms) 2024-03-21T13:08:41.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-03-21T13:08:41.946 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: hmm, no HitSet yet 2024-03-21T13:08:41.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: ok, hit_set contains 325:602f83fe:::foo:head 2024-03-21T13:08:41.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (9043 ms) 2024-03-21T13:08:41.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-03-21T13:08:41.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711026437,0 2024-03-21T13:08:41.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: first is 1711026437 2024-03-21T13:08:41.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711026437,0 2024-03-21T13:08:41.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711026437,0 2024-03-21T13:08:41.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711026437,1711026439,1711026440,0 2024-03-21T13:08:41.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711026437,1711026439,1711026440,0 2024-03-21T13:08:41.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711026437,1711026439,1711026440,0 2024-03-21T13:08:41.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711026437,1711026439,1711026440,1711026442,1711026443,0 2024-03-21T13:08:41.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711026437,1711026439,1711026440,1711026442,1711026443,0 2024-03-21T13:08:41.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711026437,1711026439,1711026440,1711026442,1711026443,0 2024-03-21T13:08:41.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711026437,1711026439,1711026440,1711026442,1711026443,1711026445,1711026446,0 2024-03-21T13:08:41.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711026437,1711026439,1711026440,1711026442,1711026443,1711026445,1711026446,0 2024-03-21T13:08:41.947 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711026437,1711026439,1711026440,1711026442,1711026443,1711026445,1711026446,0 2024-03-21T13:08:41.948 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: got ls 1711026440,1711026442,1711026443,1711026445,1711026446,1711026448,1711026449,0 2024-03-21T13:08:41.948 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: first now 1711026440, trimmed 2024-03-21T13:08:41.948 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (20445 ms) 2024-03-21T13:08:41.948 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-03-21T13:08:41.948 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: foo0 2024-03-21T13:08:41.948 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-03-21T13:08:41.948 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (14077 ms) 2024-03-21T13:08:41.948 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-03-21T13:08:41.948 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (17476 ms) 2024-03-21T13:08:41.948 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-03-21T13:08:41.948 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (23026 ms) 2024-03-21T13:08:41.948 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-03-21T13:08:41.948 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5039 ms) 2024-03-21T13:08:41.948 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-03-21T13:08:41.948 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3025 ms) 2024-03-21T13:08:41.949 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-03-21T13:08:41.949 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3022 ms) 2024-03-21T13:08:41.949 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-03-21T13:08:41.949 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3009 ms) 2024-03-21T13:08:41.949 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (231859 ms total) 2024-03-21T13:08:41.949 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: 2024-03-21T13:08:41.949 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-03-21T13:08:41.949 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (856234 ms total) 2024-03-21T13:08:41.949 INFO:tasks.workunit.client.0.smithi043.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-03-21T13:08:41.950 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:08:41.950 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81620 2024-03-21T13:08:41.950 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81620 2024-03-21T13:08:41.950 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:08:41.950 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81883 2024-03-21T13:08:41.950 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81883 2024-03-21T13:08:41.950 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:08:41.951 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82319 2024-03-21T13:08:41.951 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82319 2024-03-21T13:08:41.951 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:08:41.951 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82068 2024-03-21T13:08:41.951 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82068 2024-03-21T13:08:41.951 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:08:41.951 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81946 2024-03-21T13:08:41.951 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81946 2024-03-21T13:08:41.951 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:08:41.951 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81700 2024-03-21T13:08:41.951 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81700 2024-03-21T13:08:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:41 smithi155 ceph-mon[33846]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:08:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:41 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:08:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:41 smithi155 ceph-mon[33846]: osdmap e750: 8 total, 8 up, 8 in 2024-03-21T13:08:42.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:42.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:41 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:41 smithi043 ceph-mon[41699]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:08:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:41 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:08:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:41 smithi043 ceph-mon[41699]: osdmap e750: 8 total, 8 up, 8 in 2024-03-21T13:08:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:41 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:41 smithi043 ceph-mon[30062]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:08:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:41 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:08:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:41 smithi043 ceph-mon[30062]: osdmap e750: 8 total, 8 up, 8 in 2024-03-21T13:08:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3015684619' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:41 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-111"}]: dispatch 2024-03-21T13:08:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:42 smithi155 ceph-mon[33846]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:08:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:42 smithi155 ceph-mon[33846]: osdmap e751: 8 total, 8 up, 8 in 2024-03-21T13:08:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:42 smithi155 ceph-mon[33846]: pgmap v1175: 292 pgs: 292 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:08:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:42 smithi043 ceph-mon[30062]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:08:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:42 smithi043 ceph-mon[30062]: osdmap e751: 8 total, 8 up, 8 in 2024-03-21T13:08:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:42 smithi043 ceph-mon[30062]: pgmap v1175: 292 pgs: 292 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:08:43.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:42 smithi043 ceph-mon[41699]: from='client.36356 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi043-81880-111"}]': finished 2024-03-21T13:08:43.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:42 smithi043 ceph-mon[41699]: osdmap e751: 8 total, 8 up, 8 in 2024-03-21T13:08:43.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:42 smithi043 ceph-mon[41699]: pgmap v1175: 292 pgs: 292 active+clean; 582 KiB data, 786 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:08:45.666 INFO:tasks.workunit.client.0.smithi043.stdout:1 sec 2024-03-21T13:08:45.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.666 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.667 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.668 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.668 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.668 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.668 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.668 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.668 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.668 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.668 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.668 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.668 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.668 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.668 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.668 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.668 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.668 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.668 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.669 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.669 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.669 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.669 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.669 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.669 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.669 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.669 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.669 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.669 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.669 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.669 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.669 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.669 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.669 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.669 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.670 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.671 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.672 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:08:45.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:45 smithi155 ceph-mon[33846]: pgmap v1176: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:08:45.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:45 smithi043 ceph-mon[30062]: pgmap v1176: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:08:45.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:45 smithi043 ceph-mon[41699]: pgmap v1176: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:08:47.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:47 smithi155 ceph-mon[33846]: pgmap v1177: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 815 B/s rd, 0 op/s 2024-03-21T13:08:47.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:47 smithi043 ceph-mon[30062]: pgmap v1177: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 815 B/s rd, 0 op/s 2024-03-21T13:08:47.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:47 smithi043 ceph-mon[41699]: pgmap v1177: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 815 B/s rd, 0 op/s 2024-03-21T13:08:48.485 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:08:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:08:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:08:49.248 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:08:48 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:08:49.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:49 smithi155 ceph-mon[33846]: pgmap v1178: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:08:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:49 smithi043 ceph-mon[30062]: pgmap v1178: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:08:49.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:49 smithi043 ceph-mon[41699]: pgmap v1178: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:08:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:50 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:08:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:50 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:08:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:50 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:08:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:50 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:08:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:50 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:08:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:50 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:08:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:51 smithi155 ceph-mon[33846]: pgmap v1179: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-21T13:08:51.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:51 smithi043 ceph-mon[30062]: pgmap v1179: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-21T13:08:51.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:51 smithi043 ceph-mon[41699]: pgmap v1179: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-21T13:08:53.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:53 smithi155 ceph-mon[33846]: pgmap v1180: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 998 B/s rd, 0 op/s 2024-03-21T13:08:53.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:53 smithi043 ceph-mon[30062]: pgmap v1180: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 998 B/s rd, 0 op/s 2024-03-21T13:08:53.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:53 smithi043 ceph-mon[41699]: pgmap v1180: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 998 B/s rd, 0 op/s 2024-03-21T13:08:55.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:55 smithi155 ceph-mon[33846]: pgmap v1181: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:08:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:55 smithi043 ceph-mon[30062]: pgmap v1181: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:08:55.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:55 smithi043 ceph-mon[41699]: pgmap v1181: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:08:57.710 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:57 smithi155 ceph-mon[33846]: pgmap v1182: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:08:57.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:57 smithi043 ceph-mon[30062]: pgmap v1182: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:08:57.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:57 smithi043 ceph-mon[41699]: pgmap v1182: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:08:58.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:08:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:08:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:08:59.269 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:08:58 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:08:59.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:08:59 smithi155 ceph-mon[33846]: pgmap v1183: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:08:59.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:08:59 smithi043 ceph-mon[30062]: pgmap v1183: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:08:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:08:59 smithi043 ceph-mon[41699]: pgmap v1183: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:09:00.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:00 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:09:00.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:00 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:09:00.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:00 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:09:01.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:01 smithi155 ceph-mon[33846]: pgmap v1184: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:01.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:01 smithi043 ceph-mon[30062]: pgmap v1184: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:01.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:01 smithi043 ceph-mon[41699]: pgmap v1184: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:03.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:03 smithi155 ceph-mon[33846]: pgmap v1185: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:03.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:03 smithi043 ceph-mon[30062]: pgmap v1185: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:03.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:03 smithi043 ceph-mon[41699]: pgmap v1185: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:05.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:05 smithi155 ceph-mon[33846]: pgmap v1186: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:05.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:05 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:09:05.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:05 smithi043 ceph-mon[30062]: pgmap v1186: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:05.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:05 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:09:05.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:05 smithi043 ceph-mon[41699]: pgmap v1186: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:05.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:05 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:09:07.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:07 smithi155 ceph-mon[33846]: pgmap v1187: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:07 smithi043 ceph-mon[30062]: pgmap v1187: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:07.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:07 smithi043 ceph-mon[41699]: pgmap v1187: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:08.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:09:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:09:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:09:09.287 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:09:08 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:09:09.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:09 smithi155 ceph-mon[33846]: pgmap v1188: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:09:09.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:09 smithi043 ceph-mon[30062]: pgmap v1188: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:09:09.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:09 smithi043 ceph-mon[41699]: pgmap v1188: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:09:10.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:10 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:09:10.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:10 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:09:10.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:10 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:09:10.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:10 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:09:10.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:10 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:09:10.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:10 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:09:11.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:11 smithi155 ceph-mon[33846]: pgmap v1189: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:11.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:11 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:09:11.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:11 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:09:11.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:11 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:09:11.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:11 smithi043 ceph-mon[30062]: pgmap v1189: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:11.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:11 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:09:11.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:11 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:09:11.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:11 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:09:11.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:11 smithi043 ceph-mon[41699]: pgmap v1189: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:11.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:11 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:09:11.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:11 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:09:11.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:11 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:09:13.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:13 smithi155 ceph-mon[33846]: pgmap v1190: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:13.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:13 smithi043 ceph-mon[30062]: pgmap v1190: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:13.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:13 smithi043 ceph-mon[41699]: pgmap v1190: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:15.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:15 smithi155 ceph-mon[33846]: pgmap v1191: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:15.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:15 smithi043 ceph-mon[30062]: pgmap v1191: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:15.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:15 smithi043 ceph-mon[41699]: pgmap v1191: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:17.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:17 smithi155 ceph-mon[33846]: pgmap v1192: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:17.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:17 smithi043 ceph-mon[30062]: pgmap v1192: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:17.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:17 smithi043 ceph-mon[41699]: pgmap v1192: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:18.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:09:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:09:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:09:19.304 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:09:18 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:09:19.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:19 smithi155 ceph-mon[33846]: pgmap v1193: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:09:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:19 smithi043 ceph-mon[30062]: pgmap v1193: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:09:19.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:19 smithi043 ceph-mon[41699]: pgmap v1193: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:09:20.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:20 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:09:20.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:20 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:09:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:20 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:09:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:20 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:09:20.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:20 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:09:20.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:20 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:09:21.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:21 smithi155 ceph-mon[33846]: pgmap v1194: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:21.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:21 smithi043 ceph-mon[30062]: pgmap v1194: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:21.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:21 smithi043 ceph-mon[41699]: pgmap v1194: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:23.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:23 smithi155 ceph-mon[33846]: pgmap v1195: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:09:23.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:23 smithi043 ceph-mon[30062]: pgmap v1195: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:09:23.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:23 smithi043 ceph-mon[41699]: pgmap v1195: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:09:25.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:25 smithi155 ceph-mon[33846]: pgmap v1196: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:25.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:25 smithi043 ceph-mon[30062]: pgmap v1196: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:25.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:25 smithi043 ceph-mon[41699]: pgmap v1196: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:27.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:27 smithi155 ceph-mon[33846]: pgmap v1197: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:09:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:27 smithi043 ceph-mon[30062]: pgmap v1197: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:09:27.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:27 smithi043 ceph-mon[41699]: pgmap v1197: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:09:28.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:09:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:09:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:09:29.323 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:09:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:09:29.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:29 smithi155 ceph-mon[33846]: pgmap v1198: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:09:29.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:29 smithi043 ceph-mon[30062]: pgmap v1198: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:09:29.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:29 smithi043 ceph-mon[41699]: pgmap v1198: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:09:30.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:30 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:09:30.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:30 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:09:30.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:30 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:09:31.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:31 smithi155 ceph-mon[33846]: pgmap v1199: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:31.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:31 smithi043 ceph-mon[30062]: pgmap v1199: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:31.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:31 smithi043 ceph-mon[41699]: pgmap v1199: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:33.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:33 smithi155 ceph-mon[33846]: pgmap v1200: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:33 smithi043 ceph-mon[30062]: pgmap v1200: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:33.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:33 smithi043 ceph-mon[41699]: pgmap v1200: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:35.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:35 smithi155 ceph-mon[33846]: pgmap v1201: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:35.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:09:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:35 smithi043 ceph-mon[30062]: pgmap v1201: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:09:35.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:35 smithi043 ceph-mon[41699]: pgmap v1201: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:35.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:09:37.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:37 smithi155 ceph-mon[33846]: pgmap v1202: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:37.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:37 smithi043 ceph-mon[30062]: pgmap v1202: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:37.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:37 smithi043 ceph-mon[41699]: pgmap v1202: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:38.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:09:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:09:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:09:39.341 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:09:39 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:09:39.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:39 smithi155 ceph-mon[33846]: pgmap v1203: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:39.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:39 smithi043 ceph-mon[30062]: pgmap v1203: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:39.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:39 smithi043 ceph-mon[41699]: pgmap v1203: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:40.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:40 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:09:40.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:40 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:09:40.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:40 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:09:41.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:41 smithi155 ceph-mon[33846]: pgmap v1204: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:41.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:41 smithi043 ceph-mon[30062]: pgmap v1204: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:41 smithi043 ceph-mon[41699]: pgmap v1204: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:43 smithi155 ceph-mon[33846]: pgmap v1205: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:43 smithi043 ceph-mon[30062]: pgmap v1205: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:43.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:43 smithi043 ceph-mon[41699]: pgmap v1205: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:45.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:45 smithi155 ceph-mon[33846]: pgmap v1206: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:45 smithi043 ceph-mon[30062]: pgmap v1206: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:45.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:45 smithi043 ceph-mon[41699]: pgmap v1206: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:47.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:47 smithi155 ceph-mon[33846]: pgmap v1207: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:47.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:47 smithi043 ceph-mon[30062]: pgmap v1207: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:47.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:47 smithi043 ceph-mon[41699]: pgmap v1207: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:48.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:09:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:09:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:09:49.360 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:09:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:09:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:49 smithi155 ceph-mon[33846]: pgmap v1208: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:49 smithi043 ceph-mon[30062]: pgmap v1208: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:49.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:49 smithi043 ceph-mon[41699]: pgmap v1208: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:50 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:09:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:50 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:09:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:50 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:09:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:50 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:09:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:50 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:09:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:50 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:09:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:51 smithi155 ceph-mon[33846]: pgmap v1209: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:51.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:51 smithi043 ceph-mon[30062]: pgmap v1209: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:51.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:51 smithi043 ceph-mon[41699]: pgmap v1209: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:53.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:53 smithi043 ceph-mon[30062]: pgmap v1210: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:53.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:53 smithi043 ceph-mon[41699]: pgmap v1210: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:53 smithi155 ceph-mon[33846]: pgmap v1210: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:55.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:55 smithi043 ceph-mon[30062]: pgmap v1211: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:55.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:55 smithi043 ceph-mon[41699]: pgmap v1211: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:56.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:55 smithi155 ceph-mon[33846]: pgmap v1211: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:57.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:57 smithi043 ceph-mon[30062]: pgmap v1212: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:57.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:57 smithi043 ceph-mon[41699]: pgmap v1212: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:57 smithi155 ceph-mon[33846]: pgmap v1212: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:09:58.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:09:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:09:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:09:59.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:09:59 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:09:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:09:59 smithi043 ceph-mon[30062]: pgmap v1213: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:09:59.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:09:59 smithi043 ceph-mon[41699]: pgmap v1213: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:09:59 smithi155 ceph-mon[33846]: pgmap v1213: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:00.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:00 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:10:00.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:00 smithi043 ceph-mon[30062]: overall HEALTH_WARN 5 pool(s) do not have an application enabled 2024-03-21T13:10:00.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:00 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:10:00.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:00 smithi043 ceph-mon[41699]: overall HEALTH_WARN 5 pool(s) do not have an application enabled 2024-03-21T13:10:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:00 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:10:01.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:00 smithi155 ceph-mon[33846]: overall HEALTH_WARN 5 pool(s) do not have an application enabled 2024-03-21T13:10:01.987 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:01 smithi043 ceph-mon[30062]: pgmap v1214: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:01.987 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:01 smithi043 ceph-mon[41699]: pgmap v1214: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:01 smithi155 ceph-mon[33846]: pgmap v1214: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:03.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:03 smithi043 ceph-mon[30062]: pgmap v1215: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:03.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:03 smithi043 ceph-mon[41699]: pgmap v1215: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:03 smithi155 ceph-mon[33846]: pgmap v1215: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:05 smithi043 ceph-mon[30062]: pgmap v1216: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:05.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:05 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:10:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:05 smithi043 ceph-mon[41699]: pgmap v1216: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:05 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:10:06.009 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:05 smithi155 ceph-mon[33846]: pgmap v1216: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:06.009 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:05 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:10:08.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:07 smithi043 ceph-mon[30062]: pgmap v1217: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:08.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:07 smithi043 ceph-mon[41699]: pgmap v1217: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:07 smithi155 ceph-mon[33846]: pgmap v1217: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:08.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:10:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:10:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:10:09.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:10:09 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:10:10.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:09 smithi155 ceph-mon[33846]: pgmap v1218: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:10.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:09 smithi043 ceph-mon[30062]: pgmap v1218: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:10.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:09 smithi043 ceph-mon[41699]: pgmap v1218: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:11.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:10 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:10:11.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:10 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:10:11.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:10 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:10:11.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:10 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:10:11.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:10 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:10:11.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:10 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:10:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:11 smithi155 ceph-mon[33846]: pgmap v1219: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:11 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:10:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:11 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:10:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:11 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:10:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:11 smithi043 ceph-mon[30062]: pgmap v1219: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:11 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:10:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:11 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:10:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:11 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:10:12.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:11 smithi043 ceph-mon[41699]: pgmap v1219: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:12.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:11 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:10:12.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:11 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:10:12.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:11 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:10:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:13 smithi155 ceph-mon[33846]: pgmap v1220: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:14.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:13 smithi043 ceph-mon[30062]: pgmap v1220: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:14.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:13 smithi043 ceph-mon[41699]: pgmap v1220: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:16.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:15 smithi155 ceph-mon[33846]: pgmap v1221: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:16.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:15 smithi043 ceph-mon[30062]: pgmap v1221: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:16.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:15 smithi043 ceph-mon[41699]: pgmap v1221: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:18.025 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:17 smithi043 ceph-mon[30062]: pgmap v1222: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:18.025 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:17 smithi043 ceph-mon[41699]: pgmap v1222: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:17 smithi155 ceph-mon[33846]: pgmap v1222: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:18.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:10:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:10:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:10:19.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:10:19 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:10:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:19 smithi155 ceph-mon[33846]: pgmap v1223: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:20.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:19 smithi043 ceph-mon[30062]: pgmap v1223: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:20.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:19 smithi043 ceph-mon[41699]: pgmap v1223: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:21.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:20 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:10:21.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:20 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:10:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:20 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:10:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:20 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:10:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:20 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:10:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:20 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:10:22.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:21 smithi155 ceph-mon[33846]: pgmap v1224: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:22.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:21 smithi043 ceph-mon[30062]: pgmap v1224: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:22.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:21 smithi043 ceph-mon[41699]: pgmap v1224: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:23 smithi155 ceph-mon[33846]: pgmap v1225: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:23 smithi043 ceph-mon[30062]: pgmap v1225: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:24.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:23 smithi043 ceph-mon[41699]: pgmap v1225: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:25.678 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.678 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.678 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.678 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.679 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.679 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.679 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.679 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.679 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.679 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.679 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.679 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.679 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.679 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.679 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.679 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.679 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.679 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.679 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.680 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.680 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.680 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.680 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.680 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.680 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.680 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.680 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.680 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.680 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.680 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.680 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.680 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.680 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.680 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.680 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.681 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.681 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.681 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.681 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.681 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.681 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.681 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.681 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.681 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.681 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.681 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.681 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.681 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.681 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.681 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.681 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.682 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.682 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.682 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.682 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.682 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.682 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.682 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.682 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.683 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.683 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.683 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.683 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.683 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.683 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.683 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.683 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.683 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.683 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.683 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.683 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.683 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.683 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.683 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.683 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.684 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.684 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.684 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.684 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.684 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.684 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.684 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.684 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.684 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.684 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.684 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.684 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.684 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.684 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.684 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.685 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.685 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.685 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.685 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.685 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.685 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.685 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.685 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.685 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:25.685 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:10:26.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:25 smithi155 ceph-mon[33846]: pgmap v1226: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:26.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:25 smithi043 ceph-mon[30062]: pgmap v1226: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:26.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:25 smithi043 ceph-mon[41699]: pgmap v1226: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:28.023 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:27 smithi043 ceph-mon[30062]: pgmap v1227: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:28.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:27 smithi043 ceph-mon[41699]: pgmap v1227: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:28.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:27 smithi155 ceph-mon[33846]: pgmap v1227: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:28.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:10:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:10:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:10:29.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:10:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:10:30.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:29 smithi155 ceph-mon[33846]: pgmap v1228: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:30.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:29 smithi043 ceph-mon[30062]: pgmap v1228: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:30.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:29 smithi043 ceph-mon[41699]: pgmap v1228: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:31.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:30 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:10:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:30 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:10:31.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:30 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:10:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:31 smithi155 ceph-mon[33846]: pgmap v1229: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:31 smithi043 ceph-mon[30062]: pgmap v1229: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:31 smithi043 ceph-mon[41699]: pgmap v1229: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:34.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:33 smithi155 ceph-mon[33846]: pgmap v1230: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:10:34.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:33 smithi043 ceph-mon[30062]: pgmap v1230: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:10:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:33 smithi043 ceph-mon[41699]: pgmap v1230: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:10:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:35 smithi155 ceph-mon[33846]: pgmap v1231: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:10:36.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:35 smithi043 ceph-mon[30062]: pgmap v1231: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:36.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:10:36.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:35 smithi043 ceph-mon[41699]: pgmap v1231: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:36.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:10:38.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:37 smithi155 ceph-mon[33846]: pgmap v1232: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:38.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:37 smithi043 ceph-mon[30062]: pgmap v1232: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:38.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:10:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:10:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:10:38.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:37 smithi043 ceph-mon[41699]: pgmap v1232: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:39.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:10:39 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:10:40.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:39 smithi155 ceph-mon[33846]: pgmap v1233: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:40.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:39 smithi043 ceph-mon[30062]: pgmap v1233: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:39 smithi043 ceph-mon[41699]: pgmap v1233: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:40 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:10:41.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:40 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:10:41.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:40 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:10:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:41 smithi155 ceph-mon[33846]: pgmap v1234: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:41 smithi043 ceph-mon[30062]: pgmap v1234: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:41 smithi043 ceph-mon[41699]: pgmap v1234: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:44.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:43 smithi155 ceph-mon[33846]: pgmap v1235: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:44.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:43 smithi043 ceph-mon[30062]: pgmap v1235: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:44.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:43 smithi043 ceph-mon[41699]: pgmap v1235: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:46.131 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:45 smithi155 ceph-mon[33846]: pgmap v1236: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:46.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:45 smithi043 ceph-mon[30062]: pgmap v1236: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:46.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:45 smithi043 ceph-mon[41699]: pgmap v1236: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:47 smithi155 ceph-mon[33846]: pgmap v1237: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:47 smithi043 ceph-mon[30062]: pgmap v1237: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:48.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:10:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:10:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:10:48.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:47 smithi043 ceph-mon[41699]: pgmap v1237: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:49.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:10:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:10:50.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:49 smithi155 ceph-mon[33846]: pgmap v1238: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:49 smithi043 ceph-mon[30062]: pgmap v1238: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:50.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:49 smithi043 ceph-mon[41699]: pgmap v1238: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:50 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:10:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:50 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:10:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:50 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:10:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:50 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:10:51.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:50 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:10:51.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:50 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:10:52.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:51 smithi155 ceph-mon[33846]: pgmap v1239: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:51 smithi043 ceph-mon[30062]: pgmap v1239: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:51 smithi043 ceph-mon[41699]: pgmap v1239: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:53 smithi155 ceph-mon[33846]: pgmap v1240: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:10:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:53 smithi043 ceph-mon[30062]: pgmap v1240: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:10:54.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:53 smithi043 ceph-mon[41699]: pgmap v1240: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:10:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:55 smithi155 ceph-mon[33846]: pgmap v1241: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:56.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:55 smithi043 ceph-mon[30062]: pgmap v1241: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:56.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:55 smithi043 ceph-mon[41699]: pgmap v1241: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:10:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:57 smithi155 ceph-mon[33846]: pgmap v1242: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:58.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:57 smithi043 ceph-mon[30062]: pgmap v1242: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:58.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:10:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:10:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:10:58.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:57 smithi043 ceph-mon[41699]: pgmap v1242: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:10:59.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:10:59 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:11:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:10:59 smithi155 ceph-mon[33846]: pgmap v1243: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:10:59 smithi043 ceph-mon[30062]: pgmap v1243: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:10:59 smithi043 ceph-mon[41699]: pgmap v1243: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:00 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:11:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:00 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:11:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:00 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:11:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:01 smithi155 ceph-mon[33846]: pgmap v1244: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:01 smithi043 ceph-mon[30062]: pgmap v1244: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:01 smithi043 ceph-mon[41699]: pgmap v1244: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:03 smithi155 ceph-mon[33846]: pgmap v1245: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:11:04.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:03 smithi043 ceph-mon[30062]: pgmap v1245: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:11:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:03 smithi043 ceph-mon[41699]: pgmap v1245: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:11:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:05 smithi155 ceph-mon[33846]: pgmap v1246: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:05 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:11:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:05 smithi043 ceph-mon[30062]: pgmap v1246: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:05 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:11:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:05 smithi043 ceph-mon[41699]: pgmap v1246: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:05 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:11:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:07 smithi155 ceph-mon[33846]: pgmap v1247: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:08.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:07 smithi043 ceph-mon[30062]: pgmap v1247: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:08.235 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:11:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:11:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:11:08.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:07 smithi043 ceph-mon[41699]: pgmap v1247: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:09.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:11:09 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:11:10.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:09 smithi155 ceph-mon[33846]: pgmap v1248: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:10.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:09 smithi043 ceph-mon[30062]: pgmap v1248: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:10.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:09 smithi043 ceph-mon[41699]: pgmap v1248: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:11.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:10 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:11:11.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:10 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:11:11.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:10 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:11:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:11 smithi155 ceph-mon[33846]: pgmap v1249: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:11 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:11:12.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:11 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:11:12.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:11 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:11:12.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:11 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:11:12.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:11 smithi043 ceph-mon[30062]: pgmap v1249: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:11 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:11:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:11 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:11:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:11 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:11:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:11 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:11:12.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:11 smithi043 ceph-mon[41699]: pgmap v1249: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:12.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:11 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:11:12.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:11 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:11:12.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:11 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:11:12.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:11 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:11:13.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:12 smithi155 ceph-mon[33846]: pgmap v1250: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:13.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:12 smithi043 ceph-mon[30062]: pgmap v1250: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:13.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:12 smithi043 ceph-mon[41699]: pgmap v1250: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:15.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:15 smithi155 ceph-mon[33846]: pgmap v1251: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:15.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:15 smithi043 ceph-mon[30062]: pgmap v1251: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:15.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:15 smithi043 ceph-mon[41699]: pgmap v1251: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:17.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:17 smithi155 ceph-mon[33846]: pgmap v1252: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:17.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:17 smithi043 ceph-mon[30062]: pgmap v1252: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:17.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:17 smithi043 ceph-mon[41699]: pgmap v1252: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:18.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:11:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:11:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:11:19.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:19 smithi155 ceph-mon[33846]: pgmap v1253: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:19.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:11:19 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:11:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:19 smithi043 ceph-mon[30062]: pgmap v1253: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:19.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:19 smithi043 ceph-mon[41699]: pgmap v1253: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:20.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:20 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:11:20.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:20 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:11:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:20 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:11:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:20 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:11:20.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:20 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:11:20.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:20 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:11:21.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:21 smithi155 ceph-mon[33846]: pgmap v1254: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:21.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:21 smithi043 ceph-mon[30062]: pgmap v1254: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:21.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:21 smithi043 ceph-mon[41699]: pgmap v1254: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:23.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:23 smithi155 ceph-mon[33846]: pgmap v1255: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:11:23.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:23 smithi043 ceph-mon[30062]: pgmap v1255: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:11:23.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:23 smithi043 ceph-mon[41699]: pgmap v1255: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:11:25.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:25 smithi155 ceph-mon[33846]: pgmap v1256: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:25.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:25 smithi043 ceph-mon[30062]: pgmap v1256: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:25.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:25 smithi043 ceph-mon[41699]: pgmap v1256: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:27.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:27 smithi155 ceph-mon[33846]: pgmap v1257: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:27.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:27 smithi043 ceph-mon[30062]: pgmap v1257: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:27.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:27 smithi043 ceph-mon[41699]: pgmap v1257: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:28.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:11:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:11:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:11:29.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:29 smithi155 ceph-mon[33846]: pgmap v1258: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:29.458 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:11:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:11:29.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:29 smithi043 ceph-mon[30062]: pgmap v1258: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:29.737 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:29 smithi043 ceph-mon[41699]: pgmap v1258: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:30.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:30 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:11:30.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:30 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:11:30.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:30 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:11:31.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:31 smithi155 ceph-mon[33846]: pgmap v1259: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:31.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:31 smithi043 ceph-mon[30062]: pgmap v1259: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:31.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:31 smithi043 ceph-mon[41699]: pgmap v1259: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:33.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:33 smithi155 ceph-mon[33846]: pgmap v1260: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:11:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:33 smithi043 ceph-mon[30062]: pgmap v1260: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:11:33.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:33 smithi043 ceph-mon[41699]: pgmap v1260: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:11:35.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:35 smithi155 ceph-mon[33846]: pgmap v1261: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:35.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:11:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:35 smithi043 ceph-mon[30062]: pgmap v1261: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:11:35.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:35 smithi043 ceph-mon[41699]: pgmap v1261: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:35.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:11:37.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:37 smithi155 ceph-mon[33846]: pgmap v1262: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:37.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:37 smithi043 ceph-mon[30062]: pgmap v1262: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:37.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:37 smithi043 ceph-mon[41699]: pgmap v1262: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:38.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:11:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:11:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:11:39.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:39 smithi155 ceph-mon[33846]: pgmap v1263: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:39.458 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:11:39 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:11:39.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:39 smithi043 ceph-mon[30062]: pgmap v1263: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:39.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:39 smithi043 ceph-mon[41699]: pgmap v1263: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:40.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:40 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:11:40.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:40 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:11:40.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:40 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:11:41.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:41 smithi155 ceph-mon[33846]: pgmap v1264: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:41.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:41 smithi043 ceph-mon[30062]: pgmap v1264: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:41 smithi043 ceph-mon[41699]: pgmap v1264: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:43 smithi155 ceph-mon[33846]: pgmap v1265: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:11:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:43 smithi043 ceph-mon[30062]: pgmap v1265: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:11:43.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:43 smithi043 ceph-mon[41699]: pgmap v1265: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:11:45.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:45 smithi155 ceph-mon[33846]: pgmap v1266: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:45.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:45 smithi043 ceph-mon[30062]: pgmap v1266: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:45.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:45 smithi043 ceph-mon[41699]: pgmap v1266: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:11:47.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:47 smithi155 ceph-mon[33846]: pgmap v1267: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:47.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:47 smithi043 ceph-mon[30062]: pgmap v1267: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:47.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:47 smithi043 ceph-mon[41699]: pgmap v1267: 292 pgs: 292 active+clean; 582 KiB data, 798 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:11:48.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:11:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:11:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:11:49.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:49 smithi155 ceph-mon[33846]: pgmap v1268: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:11:49.458 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:11:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:11:49.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:49 smithi043 ceph-mon[30062]: pgmap v1268: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:11:49.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:49 smithi043 ceph-mon[41699]: pgmap v1268: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:11:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:50 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:11:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:50 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:11:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:50 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:11:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:50 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:11:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:50 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:11:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:50 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:11:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:51 smithi155 ceph-mon[33846]: pgmap v1269: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:11:51.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:51 smithi043 ceph-mon[30062]: pgmap v1269: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:11:51.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:51 smithi043 ceph-mon[41699]: pgmap v1269: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:11:53.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:53 smithi155 ceph-mon[33846]: pgmap v1270: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:11:53.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:53 smithi043 ceph-mon[30062]: pgmap v1270: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:11:53.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:53 smithi043 ceph-mon[41699]: pgmap v1270: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:11:55.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:55 smithi155 ceph-mon[33846]: pgmap v1271: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:11:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:55 smithi043 ceph-mon[30062]: pgmap v1271: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:11:55.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:55 smithi043 ceph-mon[41699]: pgmap v1271: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:11:57.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:57 smithi155 ceph-mon[33846]: pgmap v1272: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:11:57.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:57 smithi043 ceph-mon[30062]: pgmap v1272: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:11:57.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:57 smithi043 ceph-mon[41699]: pgmap v1272: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:11:58.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:11:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:11:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:11:59.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:11:59 smithi155 ceph-mon[33846]: pgmap v1273: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:11:59.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:11:59 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:11:59.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:11:59 smithi043 ceph-mon[30062]: pgmap v1273: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:11:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:11:59 smithi043 ceph-mon[41699]: pgmap v1273: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:12:00.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:00 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:12:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:00 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:12:00.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:00 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:12:01.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:01 smithi155 ceph-mon[33846]: pgmap v1274: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:01.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:01 smithi043 ceph-mon[30062]: pgmap v1274: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:01.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:01 smithi043 ceph-mon[41699]: pgmap v1274: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:03.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:03 smithi155 ceph-mon[33846]: pgmap v1275: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:12:03.736 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:03 smithi043 ceph-mon[30062]: pgmap v1275: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:12:03.736 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:03 smithi043 ceph-mon[41699]: pgmap v1275: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:12:05.690 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.691 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.691 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.691 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.691 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.691 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.691 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.691 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.691 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.691 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.691 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.691 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.691 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.691 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.691 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.692 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.692 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.692 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.692 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.692 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.692 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.692 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.692 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.692 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.692 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.692 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.692 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.692 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.692 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.693 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.693 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.693 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.693 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.693 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.693 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.693 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.693 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.693 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.693 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.693 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.693 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.693 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.693 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.693 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.694 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.694 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.694 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.694 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.694 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.694 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.694 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.694 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.694 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.694 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.694 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.694 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.694 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.694 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.694 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.695 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.695 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.695 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.695 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.695 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.695 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.695 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.695 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.695 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.695 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.695 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.695 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.695 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.695 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.695 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.696 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.696 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.696 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.696 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.696 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.696 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.696 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.696 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.696 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.696 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.696 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.696 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.696 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.696 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.696 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.697 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.697 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.697 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.697 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.697 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.697 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.697 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.697 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.697 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.697 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.697 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:12:05.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:05 smithi155 ceph-mon[33846]: pgmap v1276: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:05.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:05 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:12:05.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:05 smithi043 ceph-mon[30062]: pgmap v1276: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:05.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:05 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:12:05.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:05 smithi043 ceph-mon[41699]: pgmap v1276: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:05.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:05 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:12:07.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:07 smithi155 ceph-mon[33846]: pgmap v1277: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:07 smithi043 ceph-mon[30062]: pgmap v1277: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:07.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:07 smithi043 ceph-mon[41699]: pgmap v1277: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:08.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:12:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:12:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:12:09.405 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:12:09 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:12:09.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:09 smithi155 ceph-mon[33846]: pgmap v1278: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:09.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:09 smithi043 ceph-mon[30062]: pgmap v1278: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:09.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:09 smithi043 ceph-mon[41699]: pgmap v1278: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:10.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:10 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:12:10.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:10 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:12:10.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:10 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:12:11.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:11 smithi155 ceph-mon[33846]: pgmap v1279: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:11.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:11 smithi043 ceph-mon[30062]: pgmap v1279: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:11.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:11 smithi043 ceph-mon[41699]: pgmap v1279: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:12.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:12 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:12:12.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:12 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:12:12.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:12 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:12:12.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:12 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:12:12.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:12 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:12:12.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:12 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:12:12.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:12 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:12:12.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:12 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:12:12.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:12 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:12:12.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:12 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:12:12.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:12 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:12:12.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:12 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:12:13.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:13 smithi155 ceph-mon[33846]: pgmap v1280: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:12:13.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:13 smithi043 ceph-mon[30062]: pgmap v1280: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:12:13.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:13 smithi043 ceph-mon[41699]: pgmap v1280: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:12:15.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:15 smithi155 ceph-mon[33846]: pgmap v1281: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:15.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:15 smithi043 ceph-mon[30062]: pgmap v1281: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:15.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:15 smithi043 ceph-mon[41699]: pgmap v1281: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:17.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:17 smithi155 ceph-mon[33846]: pgmap v1282: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:17.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:17 smithi043 ceph-mon[30062]: pgmap v1282: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:17.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:17 smithi043 ceph-mon[41699]: pgmap v1282: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:18.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:12:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:12:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:12:19.432 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:12:19 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:12:19.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:19 smithi155 ceph-mon[33846]: pgmap v1283: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:19 smithi043 ceph-mon[30062]: pgmap v1283: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:19.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:19 smithi043 ceph-mon[41699]: pgmap v1283: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:20.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:20 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:12:20.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:20 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:12:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:20 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:12:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:20 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:12:20.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:20 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:12:20.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:20 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:12:21.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:21 smithi155 ceph-mon[33846]: pgmap v1284: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:21.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:21 smithi043 ceph-mon[30062]: pgmap v1284: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:21.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:21 smithi043 ceph-mon[41699]: pgmap v1284: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:23.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:23 smithi155 ceph-mon[33846]: pgmap v1285: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:12:23.735 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:23 smithi043 ceph-mon[30062]: pgmap v1285: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:12:23.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:23 smithi043 ceph-mon[41699]: pgmap v1285: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:12:25.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:25 smithi155 ceph-mon[33846]: pgmap v1286: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:25.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:25 smithi043 ceph-mon[30062]: pgmap v1286: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:25.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:25 smithi043 ceph-mon[41699]: pgmap v1286: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:27.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:27 smithi155 ceph-mon[33846]: pgmap v1287: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:27.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:27 smithi043 ceph-mon[30062]: pgmap v1287: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:27.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:27 smithi043 ceph-mon[41699]: pgmap v1287: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:28.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:12:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:12:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:12:29.452 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:12:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:12:29.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:29 smithi155 ceph-mon[33846]: pgmap v1288: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:29.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:29 smithi043 ceph-mon[30062]: pgmap v1288: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:29.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:29 smithi043 ceph-mon[41699]: pgmap v1288: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:30.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:30 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:12:30.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:30 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:12:30.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:30 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:12:31.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:31 smithi155 ceph-mon[33846]: pgmap v1289: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:31.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:31 smithi043 ceph-mon[30062]: pgmap v1289: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:31.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:31 smithi043 ceph-mon[41699]: pgmap v1289: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:33 smithi043 ceph-mon[30062]: pgmap v1290: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:12:33.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:33 smithi043 ceph-mon[41699]: pgmap v1290: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:12:33.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:33 smithi155 ceph-mon[33846]: pgmap v1290: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:12:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:35 smithi043 ceph-mon[30062]: pgmap v1291: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:12:35.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:35 smithi043 ceph-mon[41699]: pgmap v1291: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:35.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:12:35.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:35 smithi155 ceph-mon[33846]: pgmap v1291: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:35.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:12:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:37 smithi043 ceph-mon[30062]: pgmap v1292: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:37.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:37 smithi043 ceph-mon[41699]: pgmap v1292: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:37.760 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:37 smithi155 ceph-mon[33846]: pgmap v1292: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:38.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:12:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:12:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:12:39.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:12:39 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:12:39.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:39 smithi043 ceph-mon[30062]: pgmap v1293: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:39.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:39 smithi043 ceph-mon[41699]: pgmap v1293: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:39.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:39 smithi155 ceph-mon[33846]: pgmap v1293: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:40.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:40 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:12:40.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:40 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:12:40.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:40 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:12:41.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:41 smithi043 ceph-mon[30062]: pgmap v1294: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:41 smithi043 ceph-mon[41699]: pgmap v1294: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:41.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:41 smithi155 ceph-mon[33846]: pgmap v1294: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:44.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:43 smithi155 ceph-mon[33846]: pgmap v1295: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:12:44.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:43 smithi043 ceph-mon[30062]: pgmap v1295: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:12:44.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:43 smithi043 ceph-mon[41699]: pgmap v1295: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:12:46.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:45 smithi155 ceph-mon[33846]: pgmap v1296: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:46.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:45 smithi043 ceph-mon[30062]: pgmap v1296: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:46.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:45 smithi043 ceph-mon[41699]: pgmap v1296: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:48.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:47 smithi043 ceph-mon[30062]: pgmap v1297: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:48.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:47 smithi043 ceph-mon[41699]: pgmap v1297: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:48.211 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:47 smithi155 ceph-mon[33846]: pgmap v1297: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:48.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:12:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:12:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:12:49.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:12:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:12:50.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:49 smithi155 ceph-mon[33846]: pgmap v1298: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:49 smithi043 ceph-mon[30062]: pgmap v1298: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:50.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:49 smithi043 ceph-mon[41699]: pgmap v1298: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:50 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:12:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:50 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:12:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:50 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:12:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:50 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:12:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:50 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:12:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:50 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:12:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:51 smithi155 ceph-mon[33846]: pgmap v1299: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:51 smithi043 ceph-mon[30062]: pgmap v1299: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:51 smithi043 ceph-mon[41699]: pgmap v1299: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:53 smithi155 ceph-mon[33846]: pgmap v1300: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:53 smithi043 ceph-mon[30062]: pgmap v1300: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:54.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:53 smithi043 ceph-mon[41699]: pgmap v1300: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:55 smithi155 ceph-mon[33846]: pgmap v1301: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:56.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:55 smithi043 ceph-mon[30062]: pgmap v1301: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:56.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:55 smithi043 ceph-mon[41699]: pgmap v1301: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:12:58.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:57 smithi043 ceph-mon[30062]: pgmap v1302: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:58.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:57 smithi043 ceph-mon[41699]: pgmap v1302: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:57 smithi155 ceph-mon[33846]: pgmap v1302: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:12:58.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:12:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:12:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:12:59.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:12:59 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:13:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:12:59 smithi155 ceph-mon[33846]: pgmap v1303: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:00.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:12:59 smithi043 ceph-mon[30062]: pgmap v1303: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:12:59 smithi043 ceph-mon[41699]: pgmap v1303: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:00 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:13:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:00 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:13:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:00 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:13:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:01 smithi155 ceph-mon[33846]: pgmap v1304: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:01 smithi043 ceph-mon[30062]: pgmap v1304: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:01 smithi043 ceph-mon[41699]: pgmap v1304: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:03 smithi155 ceph-mon[33846]: pgmap v1305: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:03 smithi043 ceph-mon[30062]: pgmap v1305: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:03 smithi043 ceph-mon[41699]: pgmap v1305: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:05 smithi155 ceph-mon[33846]: pgmap v1306: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:05 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:13:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:05 smithi043 ceph-mon[30062]: pgmap v1306: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:05 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:13:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:05 smithi043 ceph-mon[41699]: pgmap v1306: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:05 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:13:08.035 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:07 smithi043 ceph-mon[30062]: pgmap v1307: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:08.036 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:13:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:13:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:13:08.036 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:07 smithi043 ceph-mon[41699]: pgmap v1307: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:07 smithi155 ceph-mon[33846]: pgmap v1307: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:09.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:13:09 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:13:10.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:09 smithi155 ceph-mon[33846]: pgmap v1308: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:10.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:09 smithi043 ceph-mon[30062]: pgmap v1308: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:10.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:09 smithi043 ceph-mon[41699]: pgmap v1308: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:11.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:10 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:13:11.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:10 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:13:11.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:10 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:13:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:11 smithi155 ceph-mon[33846]: pgmap v1309: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:11 smithi043 ceph-mon[30062]: pgmap v1309: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:12.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:11 smithi043 ceph-mon[41699]: pgmap v1309: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:13.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:12 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:13:13.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:12 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:13:13.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:12 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:13:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:13 smithi155 ceph-mon[33846]: pgmap v1310: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:13 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:13:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:13 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:13:14.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:13 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:13:14.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:13 smithi043 ceph-mon[30062]: pgmap v1310: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:14.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:13 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:13:14.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:13 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:13:14.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:13 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:13:14.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:13 smithi043 ceph-mon[41699]: pgmap v1310: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:14.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:13 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:13:14.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:13 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:13:14.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:13 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:13:16.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:15 smithi155 ceph-mon[33846]: pgmap v1311: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:16.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:15 smithi043 ceph-mon[30062]: pgmap v1311: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:16.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:15 smithi043 ceph-mon[41699]: pgmap v1311: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:17.457 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 13:13:17 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=cleanup t=2024-03-21T13:13:17.149855173Z level=info msg="Completed cleanup jobs" duration=3.242355ms 2024-03-21T13:13:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:17 smithi155 ceph-mon[33846]: pgmap v1312: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:18.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:17 smithi043 ceph-mon[30062]: pgmap v1312: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:18.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:13:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:13:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:13:18.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:17 smithi043 ceph-mon[41699]: pgmap v1312: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:19.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:13:19 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:13:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:19 smithi155 ceph-mon[33846]: pgmap v1313: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:20.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:19 smithi043 ceph-mon[30062]: pgmap v1313: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:20.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:19 smithi043 ceph-mon[41699]: pgmap v1313: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:20 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:13:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:20 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:13:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:20 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:13:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:20 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:13:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:20 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:13:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:20 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:13:22.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:21 smithi155 ceph-mon[33846]: pgmap v1314: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:22.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:21 smithi043 ceph-mon[30062]: pgmap v1314: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:22.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:21 smithi043 ceph-mon[41699]: pgmap v1314: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:23 smithi155 ceph-mon[33846]: pgmap v1315: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:13:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:23 smithi043 ceph-mon[30062]: pgmap v1315: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:13:24.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:23 smithi043 ceph-mon[41699]: pgmap v1315: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:13:26.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:25 smithi155 ceph-mon[33846]: pgmap v1316: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:26.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:25 smithi043 ceph-mon[30062]: pgmap v1316: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:26.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:25 smithi043 ceph-mon[41699]: pgmap v1316: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:27 smithi155 ceph-mon[33846]: pgmap v1317: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:28.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:27 smithi043 ceph-mon[30062]: pgmap v1317: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:28.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:13:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:13:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:13:28.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:27 smithi043 ceph-mon[41699]: pgmap v1317: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:29.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:13:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:13:30.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:29 smithi155 ceph-mon[33846]: pgmap v1318: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:29 smithi043 ceph-mon[30062]: pgmap v1318: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:30.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:29 smithi043 ceph-mon[41699]: pgmap v1318: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:31.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:30 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:13:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:30 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:13:31.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:30 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:13:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:31 smithi155 ceph-mon[33846]: pgmap v1319: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:31 smithi043 ceph-mon[30062]: pgmap v1319: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:31 smithi043 ceph-mon[41699]: pgmap v1319: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:34.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:33 smithi155 ceph-mon[33846]: pgmap v1320: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:34.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:33 smithi043 ceph-mon[30062]: pgmap v1320: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:33 smithi043 ceph-mon[41699]: pgmap v1320: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:35 smithi155 ceph-mon[33846]: pgmap v1321: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:13:36.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:35 smithi043 ceph-mon[30062]: pgmap v1321: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:36.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:13:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:35 smithi043 ceph-mon[41699]: pgmap v1321: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:13:38.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:37 smithi155 ceph-mon[33846]: pgmap v1322: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:38.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:37 smithi043 ceph-mon[30062]: pgmap v1322: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:38.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:13:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:13:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:13:38.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:37 smithi043 ceph-mon[41699]: pgmap v1322: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:39.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:13:39 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:13:40.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:39 smithi155 ceph-mon[33846]: pgmap v1323: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:40.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:39 smithi043 ceph-mon[30062]: pgmap v1323: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:39 smithi043 ceph-mon[41699]: pgmap v1323: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:40 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:13:41.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:40 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:13:41.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:40 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:13:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:41 smithi155 ceph-mon[33846]: pgmap v1324: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:41 smithi043 ceph-mon[30062]: pgmap v1324: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:41 smithi043 ceph-mon[41699]: pgmap v1324: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:44.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:43 smithi155 ceph-mon[33846]: pgmap v1325: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:44.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:43 smithi043 ceph-mon[30062]: pgmap v1325: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:44.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:43 smithi043 ceph-mon[41699]: pgmap v1325: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:45.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.703 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.704 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.705 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.705 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.705 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.705 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.705 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.705 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.705 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.705 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.705 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.705 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.705 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.705 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.705 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.705 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.705 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.706 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.706 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.706 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.706 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.706 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.706 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.706 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.706 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.706 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.706 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.706 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.706 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.706 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.706 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.707 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.708 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.708 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.708 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.708 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.708 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.708 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.708 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.708 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.708 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.708 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.708 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.708 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.708 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.708 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:45.709 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:13:46.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:45 smithi155 ceph-mon[33846]: pgmap v1326: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:46.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:45 smithi043 ceph-mon[30062]: pgmap v1326: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:46.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:45 smithi043 ceph-mon[41699]: pgmap v1326: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:47 smithi155 ceph-mon[33846]: pgmap v1327: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:47 smithi043 ceph-mon[30062]: pgmap v1327: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:48.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:13:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:13:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:13:48.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:47 smithi043 ceph-mon[41699]: pgmap v1327: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:49.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:13:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:13:50.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:49 smithi155 ceph-mon[33846]: pgmap v1328: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:49 smithi043 ceph-mon[30062]: pgmap v1328: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:50.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:49 smithi043 ceph-mon[41699]: pgmap v1328: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:50 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:13:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:50 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:13:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:50 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:13:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:50 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:13:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:50 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:13:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:50 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:13:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:51 smithi155 ceph-mon[33846]: pgmap v1329: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:51 smithi043 ceph-mon[30062]: pgmap v1329: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:51 smithi043 ceph-mon[41699]: pgmap v1329: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:53 smithi155 ceph-mon[33846]: pgmap v1330: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:13:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:53 smithi043 ceph-mon[30062]: pgmap v1330: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:13:54.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:53 smithi043 ceph-mon[41699]: pgmap v1330: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:13:56.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:55 smithi155 ceph-mon[33846]: pgmap v1331: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:56.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:55 smithi043 ceph-mon[30062]: pgmap v1331: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:56.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:55 smithi043 ceph-mon[41699]: pgmap v1331: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:13:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:57 smithi155 ceph-mon[33846]: pgmap v1332: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:58.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:57 smithi043 ceph-mon[30062]: pgmap v1332: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:58.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:13:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:13:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:13:58.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:57 smithi043 ceph-mon[41699]: pgmap v1332: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:13:59.708 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:13:59 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:14:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:13:59 smithi155 ceph-mon[33846]: pgmap v1333: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:13:59 smithi043 ceph-mon[30062]: pgmap v1333: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:13:59 smithi043 ceph-mon[41699]: pgmap v1333: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:00 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:14:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:00 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:14:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:00 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:14:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:01 smithi155 ceph-mon[33846]: pgmap v1334: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:01 smithi043 ceph-mon[30062]: pgmap v1334: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:01 smithi043 ceph-mon[41699]: pgmap v1334: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:02 smithi155 ceph-mon[33846]: pgmap v1335: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:14:03.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:02 smithi043 ceph-mon[30062]: pgmap v1335: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:14:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:02 smithi043 ceph-mon[41699]: pgmap v1335: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:14:05.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:05 smithi155 ceph-mon[33846]: pgmap v1336: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:05.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:05 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:14:05.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:05 smithi043 ceph-mon[30062]: pgmap v1336: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:05.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:05 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:14:05.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:05 smithi043 ceph-mon[41699]: pgmap v1336: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:05.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:05 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:14:07.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:07 smithi155 ceph-mon[33846]: pgmap v1337: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:07 smithi043 ceph-mon[30062]: pgmap v1337: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:07.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:07 smithi043 ceph-mon[41699]: pgmap v1337: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:08.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:14:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:14:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:14:09.510 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:09 smithi155 ceph-mon[33846]: pgmap v1338: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:09.510 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:14:09 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:14:09.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:09 smithi043 ceph-mon[30062]: pgmap v1338: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:09.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:09 smithi043 ceph-mon[41699]: pgmap v1338: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:10.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:10 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:14:10.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:10 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:14:10.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:10 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:14:11.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:11 smithi155 ceph-mon[33846]: pgmap v1339: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:11.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:11 smithi043 ceph-mon[30062]: pgmap v1339: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:11.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:11 smithi043 ceph-mon[41699]: pgmap v1339: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:13.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:13 smithi155 ceph-mon[33846]: pgmap v1340: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:14:13.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:13 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:14:13.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:13 smithi043 ceph-mon[30062]: pgmap v1340: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:14:13.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:13 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:14:13.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:13 smithi043 ceph-mon[41699]: pgmap v1340: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:14:13.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:13 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:14:14.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:14 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:14:14.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:14 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:14:14.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:14 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:14:14.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:14 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:14:14.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:14 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:14:14.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:14 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:14:14.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:14 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:14:14.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:14 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:14:14.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:14 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:14:16.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:15 smithi155 ceph-mon[33846]: pgmap v1341: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:16.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:15 smithi043 ceph-mon[30062]: pgmap v1341: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:16.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:15 smithi043 ceph-mon[41699]: pgmap v1341: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:17 smithi155 ceph-mon[33846]: pgmap v1342: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:18.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:17 smithi043 ceph-mon[30062]: pgmap v1342: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:18.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:14:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:14:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:14:18.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:17 smithi043 ceph-mon[41699]: pgmap v1342: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:19.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:14:19 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:14:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:19 smithi155 ceph-mon[33846]: pgmap v1343: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:20.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:19 smithi043 ceph-mon[30062]: pgmap v1343: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:20.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:19 smithi043 ceph-mon[41699]: pgmap v1343: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:20 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:14:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:20 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:14:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:20 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:14:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:20 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:14:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:20 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:14:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:20 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:14:22.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:21 smithi155 ceph-mon[33846]: pgmap v1344: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:22.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:21 smithi043 ceph-mon[30062]: pgmap v1344: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:22.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:21 smithi043 ceph-mon[41699]: pgmap v1344: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:23 smithi155 ceph-mon[33846]: pgmap v1345: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:23 smithi043 ceph-mon[30062]: pgmap v1345: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:24.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:23 smithi043 ceph-mon[41699]: pgmap v1345: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:26.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:25 smithi155 ceph-mon[33846]: pgmap v1346: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:26.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:25 smithi043 ceph-mon[30062]: pgmap v1346: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:26.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:25 smithi043 ceph-mon[41699]: pgmap v1346: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:27 smithi155 ceph-mon[33846]: pgmap v1347: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:28.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:27 smithi043 ceph-mon[30062]: pgmap v1347: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:28.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:14:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:14:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:14:28.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:27 smithi043 ceph-mon[41699]: pgmap v1347: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:29.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:14:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:14:30.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:29 smithi155 ceph-mon[33846]: pgmap v1348: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:30.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:29 smithi043 ceph-mon[30062]: pgmap v1348: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:30.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:29 smithi043 ceph-mon[41699]: pgmap v1348: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:31.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:30 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:14:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:30 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:14:31.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:30 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:14:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:31 smithi155 ceph-mon[33846]: pgmap v1349: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:31 smithi043 ceph-mon[30062]: pgmap v1349: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:31 smithi043 ceph-mon[41699]: pgmap v1349: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:34.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:33 smithi155 ceph-mon[33846]: pgmap v1350: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:34.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:33 smithi043 ceph-mon[30062]: pgmap v1350: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:33 smithi043 ceph-mon[41699]: pgmap v1350: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:35 smithi155 ceph-mon[33846]: pgmap v1351: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:36.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:14:36.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:35 smithi043 ceph-mon[30062]: pgmap v1351: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:36.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:14:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:35 smithi043 ceph-mon[41699]: pgmap v1351: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:14:38.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:37 smithi155 ceph-mon[33846]: pgmap v1352: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:38.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:37 smithi043 ceph-mon[30062]: pgmap v1352: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:38.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:14:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:14:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:14:38.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:37 smithi043 ceph-mon[41699]: pgmap v1352: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:39.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:14:39 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:14:40.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:39 smithi155 ceph-mon[33846]: pgmap v1353: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:40.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:39 smithi043 ceph-mon[30062]: pgmap v1353: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:39 smithi043 ceph-mon[41699]: pgmap v1353: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:40 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:14:41.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:40 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:14:41.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:40 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:14:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:41 smithi155 ceph-mon[33846]: pgmap v1354: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:41 smithi043 ceph-mon[30062]: pgmap v1354: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:41 smithi043 ceph-mon[41699]: pgmap v1354: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:44.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:43 smithi155 ceph-mon[33846]: pgmap v1355: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:14:44.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:43 smithi043 ceph-mon[30062]: pgmap v1355: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:14:44.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:43 smithi043 ceph-mon[41699]: pgmap v1355: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:14:46.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:45 smithi155 ceph-mon[33846]: pgmap v1356: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:46.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:45 smithi043 ceph-mon[30062]: pgmap v1356: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:46.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:45 smithi043 ceph-mon[41699]: pgmap v1356: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:47 smithi155 ceph-mon[33846]: pgmap v1357: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:47 smithi043 ceph-mon[30062]: pgmap v1357: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:48.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:47 smithi043 ceph-mon[41699]: pgmap v1357: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:48.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:14:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:14:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:14:49.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:14:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:14:50.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:49 smithi155 ceph-mon[33846]: pgmap v1358: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:50.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:49 smithi043 ceph-mon[30062]: pgmap v1358: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:50.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:49 smithi043 ceph-mon[41699]: pgmap v1358: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:50 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:14:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:50 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:14:51.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:50 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:14:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:50 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:14:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:50 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:14:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:50 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:14:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:51 smithi155 ceph-mon[33846]: pgmap v1359: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:51 smithi043 ceph-mon[30062]: pgmap v1359: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:51 smithi043 ceph-mon[41699]: pgmap v1359: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:53.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:52 smithi155 ceph-mon[33846]: pgmap v1360: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:14:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:52 smithi043 ceph-mon[30062]: pgmap v1360: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:14:53.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:52 smithi043 ceph-mon[41699]: pgmap v1360: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:14:55.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:55 smithi155 ceph-mon[33846]: pgmap v1361: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:55.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:55 smithi043 ceph-mon[30062]: pgmap v1361: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:55.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:55 smithi043 ceph-mon[41699]: pgmap v1361: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:57.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:57 smithi155 ceph-mon[33846]: pgmap v1362: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:57.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:57 smithi043 ceph-mon[41699]: pgmap v1362: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:57.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:57 smithi043 ceph-mon[30062]: pgmap v1362: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:14:58.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:14:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:14:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:14:59.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:14:59 smithi155 ceph-mon[33846]: pgmap v1363: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:59.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:14:59 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:14:59.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:14:59 smithi043 ceph-mon[30062]: pgmap v1363: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:14:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:14:59 smithi043 ceph-mon[41699]: pgmap v1363: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:00.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:00 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:15:00.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:00 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:15:00.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:00 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:15:01.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:01 smithi155 ceph-mon[33846]: pgmap v1364: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:01.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:01 smithi043 ceph-mon[30062]: pgmap v1364: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:01.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:01 smithi043 ceph-mon[41699]: pgmap v1364: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:03.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:03 smithi155 ceph-mon[33846]: pgmap v1365: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:03.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:03 smithi043 ceph-mon[30062]: pgmap v1365: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:03.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:03 smithi043 ceph-mon[41699]: pgmap v1365: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:05.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:05 smithi155 ceph-mon[33846]: pgmap v1366: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:05.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:05 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:15:05.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:05 smithi043 ceph-mon[30062]: pgmap v1366: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:05.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:05 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:15:05.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:05 smithi043 ceph-mon[41699]: pgmap v1366: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:05.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:05 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:15:07.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:07 smithi155 ceph-mon[33846]: pgmap v1367: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:07.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:07 smithi043 ceph-mon[30062]: pgmap v1367: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:07.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:07 smithi043 ceph-mon[41699]: pgmap v1367: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:08.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:15:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:15:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:15:09.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:09 smithi155 ceph-mon[33846]: pgmap v1368: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:09.708 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:15:09 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:15:09.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:09 smithi043 ceph-mon[30062]: pgmap v1368: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:09.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:09 smithi043 ceph-mon[41699]: pgmap v1368: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:10.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:10 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:15:10.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:10 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:15:10.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:10 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:15:11.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:11 smithi155 ceph-mon[33846]: pgmap v1369: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:11.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:11 smithi043 ceph-mon[30062]: pgmap v1369: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:11.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:11 smithi043 ceph-mon[41699]: pgmap v1369: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:13.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:13 smithi155 ceph-mon[33846]: pgmap v1370: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:15:13.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:13 smithi043 ceph-mon[30062]: pgmap v1370: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:15:13.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:13 smithi043 ceph-mon[41699]: pgmap v1370: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:15:14.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:14 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:15:14.711 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:14 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:15:14.711 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:14 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:15:15.690 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:15 smithi155 ceph-mon[33846]: pgmap v1371: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:15.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:15 smithi043 ceph-mon[30062]: pgmap v1371: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:15.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:15 smithi043 ceph-mon[41699]: pgmap v1371: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:17.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:17 smithi155 ceph-mon[33846]: pgmap v1372: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:15:17.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:17 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:15:17.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:17 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:15:17.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:17 smithi043 ceph-mon[30062]: pgmap v1372: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:15:17.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:17 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:15:17.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:17 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:15:17.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:17 smithi043 ceph-mon[41699]: pgmap v1372: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:15:17.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:17 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:15:17.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:17 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:15:18.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:15:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:15:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:15:19.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:15:19 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:15:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:19 smithi155 ceph-mon[33846]: pgmap v1373: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:19 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:15:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:19 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:15:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:19 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:15:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:19 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:15:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:19 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:15:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:19 smithi043 ceph-mon[30062]: pgmap v1373: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:19 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:15:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:19 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:15:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:19 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:15:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:19 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:15:19.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:19 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:15:19.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:19 smithi043 ceph-mon[41699]: pgmap v1373: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:19.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:19 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:15:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:19 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:15:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:19 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:15:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:19 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:15:19.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:19 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:15:20.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:20 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:15:20.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:20 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:15:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:20 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:15:20.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:20 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:15:20.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:20 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:15:20.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:20 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:15:21.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:21 smithi155 ceph-mon[33846]: pgmap v1374: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:15:21.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:21 smithi043 ceph-mon[30062]: pgmap v1374: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:15:21.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:21 smithi043 ceph-mon[41699]: pgmap v1374: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:15:23.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:23 smithi155 ceph-mon[33846]: pgmap v1375: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:15:23.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:23 smithi043 ceph-mon[30062]: pgmap v1375: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:15:23.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:23 smithi043 ceph-mon[41699]: pgmap v1375: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:15:25.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:25 smithi155 ceph-mon[33846]: pgmap v1376: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:25.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.715 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.716 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.717 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.718 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.719 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.720 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.721 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.722 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.722 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.722 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.722 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.722 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.722 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.722 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.722 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:15:25.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:25 smithi043 ceph-mon[30062]: pgmap v1376: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:25.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:25 smithi043 ceph-mon[41699]: pgmap v1376: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:27.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:27 smithi155 ceph-mon[33846]: pgmap v1377: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:15:27.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:27 smithi043 ceph-mon[30062]: pgmap v1377: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:15:27.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:27 smithi043 ceph-mon[41699]: pgmap v1377: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:15:28.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:15:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:15:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:15:29.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:29 smithi155 ceph-mon[33846]: pgmap v1378: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:29.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:15:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:15:29.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:29 smithi043 ceph-mon[30062]: pgmap v1378: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:29.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:29 smithi043 ceph-mon[41699]: pgmap v1378: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:30.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:30 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:15:30.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:30 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:15:30.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:30 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:15:31.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:31 smithi155 ceph-mon[33846]: pgmap v1379: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:31.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:31 smithi043 ceph-mon[30062]: pgmap v1379: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:31.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:31 smithi043 ceph-mon[41699]: pgmap v1379: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:33.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:33 smithi155 ceph-mon[33846]: pgmap v1380: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:15:33.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:33 smithi043 ceph-mon[30062]: pgmap v1380: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:15:33.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:33 smithi043 ceph-mon[41699]: pgmap v1380: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:15:35.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:35 smithi155 ceph-mon[33846]: pgmap v1381: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:35.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:15:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:35 smithi043 ceph-mon[30062]: pgmap v1381: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:35.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:15:35.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:35 smithi043 ceph-mon[41699]: pgmap v1381: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:35.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:15:37.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:37 smithi155 ceph-mon[33846]: pgmap v1382: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:37.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:37 smithi043 ceph-mon[30062]: pgmap v1382: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:37.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:37 smithi043 ceph-mon[41699]: pgmap v1382: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:38.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:15:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:15:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:15:39.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:39 smithi155 ceph-mon[33846]: pgmap v1383: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:39.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:15:39 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:15:39.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:39 smithi043 ceph-mon[30062]: pgmap v1383: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:39.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:39 smithi043 ceph-mon[41699]: pgmap v1383: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:40.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:40 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:15:40.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:40 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:15:40.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:40 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:15:41.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:41 smithi155 ceph-mon[33846]: pgmap v1384: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:41.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:41 smithi043 ceph-mon[30062]: pgmap v1384: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:41.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:41 smithi043 ceph-mon[41699]: pgmap v1384: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:43.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:43 smithi155 ceph-mon[33846]: pgmap v1385: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:43.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:43 smithi043 ceph-mon[30062]: pgmap v1385: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:43.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:43 smithi043 ceph-mon[41699]: pgmap v1385: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:45.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:45 smithi155 ceph-mon[33846]: pgmap v1386: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:45.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:45 smithi043 ceph-mon[30062]: pgmap v1386: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:45.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:45 smithi043 ceph-mon[41699]: pgmap v1386: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:47.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:47 smithi155 ceph-mon[33846]: pgmap v1387: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:47.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:47 smithi043 ceph-mon[30062]: pgmap v1387: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:47.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:47 smithi043 ceph-mon[41699]: pgmap v1387: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:48.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:15:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:15:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:15:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:49 smithi155 ceph-mon[33846]: pgmap v1388: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:49.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:15:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:15:49.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:49 smithi043 ceph-mon[30062]: pgmap v1388: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:49.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:49 smithi043 ceph-mon[41699]: pgmap v1388: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:50 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:15:50.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:50 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:15:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:50 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:15:50.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:50 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:15:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:50 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:15:50.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:50 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:15:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:51 smithi155 ceph-mon[33846]: pgmap v1389: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:51.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:51 smithi043 ceph-mon[30062]: pgmap v1389: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:51.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:51 smithi043 ceph-mon[41699]: pgmap v1389: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:53.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:53 smithi155 ceph-mon[33846]: pgmap v1390: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:15:53.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:53 smithi043 ceph-mon[30062]: pgmap v1390: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:15:53.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:53 smithi043 ceph-mon[41699]: pgmap v1390: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:15:55.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:55 smithi155 ceph-mon[33846]: pgmap v1391: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:55.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:55 smithi043 ceph-mon[30062]: pgmap v1391: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:55.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:55 smithi043 ceph-mon[41699]: pgmap v1391: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:57.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:57 smithi155 ceph-mon[33846]: pgmap v1392: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:57.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:57 smithi043 ceph-mon[30062]: pgmap v1392: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:57.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:57 smithi043 ceph-mon[41699]: pgmap v1392: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:15:58.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:15:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:15:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:15:59.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:15:59 smithi155 ceph-mon[33846]: pgmap v1393: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:59.708 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:15:59 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:15:59.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:15:59 smithi043 ceph-mon[30062]: pgmap v1393: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:15:59.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:15:59 smithi043 ceph-mon[41699]: pgmap v1393: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:00.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:00 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:16:00.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:00 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:16:00.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:00 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:16:01.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:01 smithi043 ceph-mon[30062]: pgmap v1394: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:01.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:01 smithi043 ceph-mon[41699]: pgmap v1394: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:01.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:01 smithi155 ceph-mon[33846]: pgmap v1394: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:03.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:03 smithi043 ceph-mon[30062]: pgmap v1395: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:16:03.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:03 smithi043 ceph-mon[41699]: pgmap v1395: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:16:03.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:03 smithi155 ceph-mon[33846]: pgmap v1395: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:16:05.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:05 smithi043 ceph-mon[41699]: pgmap v1396: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:05.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:05 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:16:05.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:05 smithi043 ceph-mon[30062]: pgmap v1396: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:05.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:05 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:16:05.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:05 smithi155 ceph-mon[33846]: pgmap v1396: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:05.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:05 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:16:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:07 smithi043 ceph-mon[30062]: pgmap v1397: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:07.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:07 smithi043 ceph-mon[41699]: pgmap v1397: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:07.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:07 smithi155 ceph-mon[33846]: pgmap v1397: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:08.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:16:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:16:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:16:09.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:09 smithi155 ceph-mon[33846]: pgmap v1398: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:09.708 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:16:09 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:16:09.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:09 smithi043 ceph-mon[30062]: pgmap v1398: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:09.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:09 smithi043 ceph-mon[41699]: pgmap v1398: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:10 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:16:10.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:10 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:16:10.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:10 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:16:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:11 smithi155 ceph-mon[33846]: pgmap v1399: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:11.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:11 smithi043 ceph-mon[30062]: pgmap v1399: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:11.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:11 smithi043 ceph-mon[41699]: pgmap v1399: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:13 smithi155 ceph-mon[33846]: pgmap v1400: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:16:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:13 smithi043 ceph-mon[30062]: pgmap v1400: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:16:13.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:13 smithi043 ceph-mon[41699]: pgmap v1400: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:16:15.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:15 smithi155 ceph-mon[33846]: pgmap v1401: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:15.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:15 smithi043 ceph-mon[30062]: pgmap v1401: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:15.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:15 smithi043 ceph-mon[41699]: pgmap v1401: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:17.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:17 smithi155 ceph-mon[33846]: pgmap v1402: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:17 smithi043 ceph-mon[30062]: pgmap v1402: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:17.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:17 smithi043 ceph-mon[41699]: pgmap v1402: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:18.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:16:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:16:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:16:19.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:16:19 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:16:19.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:19 smithi155 ceph-mon[33846]: pgmap v1403: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:19.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:19 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:16:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:19 smithi043 ceph-mon[30062]: pgmap v1403: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:19 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:16:19.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:19 smithi043 ceph-mon[41699]: pgmap v1403: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:19.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:19 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:16:20.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:20 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:16:20.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:20 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:16:20.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:20 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:16:20.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:20 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:16:20.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:20 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:16:20.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:20 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:16:20.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:20 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:16:20.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:20 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:16:20.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:20 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:16:20.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:20 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:16:20.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:20 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:16:20.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:20 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:16:20.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:20 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:16:20.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:20 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:16:20.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:20 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:16:22.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:21 smithi155 ceph-mon[33846]: pgmap v1404: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:22.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:21 smithi043 ceph-mon[30062]: pgmap v1404: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:22.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:21 smithi043 ceph-mon[41699]: pgmap v1404: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:23 smithi155 ceph-mon[33846]: pgmap v1405: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:16:24.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:23 smithi043 ceph-mon[30062]: pgmap v1405: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:16:24.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:23 smithi043 ceph-mon[41699]: pgmap v1405: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:16:26.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:25 smithi155 ceph-mon[33846]: pgmap v1406: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:26.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:25 smithi043 ceph-mon[30062]: pgmap v1406: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:26.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:25 smithi043 ceph-mon[41699]: pgmap v1406: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:28.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:27 smithi043 ceph-mon[30062]: pgmap v1407: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:28.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:27 smithi043 ceph-mon[41699]: pgmap v1407: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:27 smithi155 ceph-mon[33846]: pgmap v1407: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:28.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:16:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:16:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:16:29.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:16:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:16:30.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:29 smithi155 ceph-mon[33846]: pgmap v1408: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:29 smithi043 ceph-mon[30062]: pgmap v1408: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:30.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:29 smithi043 ceph-mon[41699]: pgmap v1408: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:31.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:30 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:16:31.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:30 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:16:31.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:30 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:16:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:31 smithi155 ceph-mon[33846]: pgmap v1409: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:32.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:31 smithi043 ceph-mon[30062]: pgmap v1409: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:31 smithi043 ceph-mon[41699]: pgmap v1409: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:34.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:33 smithi155 ceph-mon[33846]: pgmap v1410: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:34.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:33 smithi043 ceph-mon[30062]: pgmap v1410: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:33 smithi043 ceph-mon[41699]: pgmap v1410: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:35 smithi155 ceph-mon[33846]: pgmap v1411: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:16:36.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:35 smithi043 ceph-mon[30062]: pgmap v1411: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:36.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:16:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:35 smithi043 ceph-mon[41699]: pgmap v1411: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:16:38.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:37 smithi043 ceph-mon[30062]: pgmap v1412: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:38.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:37 smithi043 ceph-mon[41699]: pgmap v1412: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:38.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:37 smithi155 ceph-mon[33846]: pgmap v1412: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:38.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:16:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:16:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:16:39.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:16:39 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:16:40.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:39 smithi155 ceph-mon[33846]: pgmap v1413: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:40.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:39 smithi043 ceph-mon[30062]: pgmap v1413: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:39 smithi043 ceph-mon[41699]: pgmap v1413: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:40.985 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:40 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:16:41.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:40 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:16:41.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:40 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:16:42.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:41 smithi155 ceph-mon[33846]: pgmap v1414: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:42.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:41 smithi043 ceph-mon[30062]: pgmap v1414: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:41 smithi043 ceph-mon[41699]: pgmap v1414: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:44.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:43 smithi155 ceph-mon[33846]: pgmap v1415: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:44.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:43 smithi043 ceph-mon[30062]: pgmap v1415: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:44.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:43 smithi043 ceph-mon[41699]: pgmap v1415: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:46.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:45 smithi155 ceph-mon[33846]: pgmap v1416: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:46.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:45 smithi043 ceph-mon[30062]: pgmap v1416: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:46.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:45 smithi043 ceph-mon[41699]: pgmap v1416: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:47 smithi155 ceph-mon[33846]: pgmap v1417: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:47 smithi043 ceph-mon[30062]: pgmap v1417: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:48.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:16:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:16:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:16:48.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:47 smithi043 ceph-mon[41699]: pgmap v1417: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:49.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:16:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:16:50.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:49 smithi155 ceph-mon[33846]: pgmap v1418: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:49 smithi043 ceph-mon[30062]: pgmap v1418: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:50.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:49 smithi043 ceph-mon[41699]: pgmap v1418: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:50 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:16:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:50 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:16:51.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:50 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:16:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:50 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:16:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:50 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:16:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:50 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:16:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:51 smithi155 ceph-mon[33846]: pgmap v1419: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:52.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:51 smithi043 ceph-mon[30062]: pgmap v1419: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:51 smithi043 ceph-mon[41699]: pgmap v1419: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:53 smithi155 ceph-mon[33846]: pgmap v1420: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:16:54.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:53 smithi043 ceph-mon[30062]: pgmap v1420: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:16:54.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:53 smithi043 ceph-mon[41699]: pgmap v1420: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:16:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:55 smithi155 ceph-mon[33846]: pgmap v1421: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:56.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:55 smithi043 ceph-mon[30062]: pgmap v1421: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:56.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:55 smithi043 ceph-mon[41699]: pgmap v1421: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:16:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:57 smithi155 ceph-mon[33846]: pgmap v1422: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:58.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:57 smithi043 ceph-mon[30062]: pgmap v1422: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:58.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:16:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:16:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:16:58.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:57 smithi043 ceph-mon[41699]: pgmap v1422: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:16:59.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:16:59 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:17:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:16:59 smithi155 ceph-mon[33846]: pgmap v1423: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:00.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:16:59 smithi043 ceph-mon[30062]: pgmap v1423: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:16:59 smithi043 ceph-mon[41699]: pgmap v1423: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:00 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:17:01.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:00 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:17:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:00 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:17:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:01 smithi155 ceph-mon[33846]: pgmap v1424: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:01 smithi043 ceph-mon[30062]: pgmap v1424: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:01 smithi043 ceph-mon[41699]: pgmap v1424: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:03 smithi155 ceph-mon[33846]: pgmap v1425: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:17:04.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:03 smithi043 ceph-mon[30062]: pgmap v1425: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:17:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:03 smithi043 ceph-mon[41699]: pgmap v1425: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:17:05.727 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.728 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.728 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.728 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.728 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.728 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.728 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.728 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.728 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.728 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.728 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.728 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.728 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.728 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.728 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.729 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.729 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.729 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.729 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.729 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.729 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.729 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.729 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.729 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.729 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.729 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.729 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.729 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.729 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.730 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.730 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.730 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.730 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.730 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.730 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.730 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.730 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.730 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.730 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.730 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.730 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.730 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.730 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.731 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.731 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.731 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.731 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.731 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.731 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.731 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.731 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.731 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.731 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.731 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.731 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.731 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.731 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.731 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.732 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.732 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.732 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.732 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.732 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.732 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.732 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.732 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.732 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.732 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.732 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.732 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.732 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.732 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.733 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.733 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.733 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.733 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.733 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.733 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.733 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.733 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.733 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.733 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.733 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.733 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.733 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.733 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:05.734 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:17:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:05 smithi155 ceph-mon[33846]: pgmap v1426: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:05 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:17:06.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:05 smithi043 ceph-mon[30062]: pgmap v1426: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:06.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:05 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:17:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:05 smithi043 ceph-mon[41699]: pgmap v1426: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:05 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:17:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:07 smithi155 ceph-mon[33846]: pgmap v1427: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:08.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:07 smithi043 ceph-mon[30062]: pgmap v1427: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:08.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:07 smithi043 ceph-mon[41699]: pgmap v1427: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:08.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:17:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:17:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:17:09.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:17:09 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:17:10.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:09 smithi155 ceph-mon[33846]: pgmap v1428: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:10.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:09 smithi043 ceph-mon[30062]: pgmap v1428: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:10.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:09 smithi043 ceph-mon[41699]: pgmap v1428: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:11.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:10 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:17:11.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:10 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:17:11.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:10 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:17:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:11 smithi155 ceph-mon[33846]: pgmap v1429: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:11 smithi043 ceph-mon[30062]: pgmap v1429: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:12.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:11 smithi043 ceph-mon[41699]: pgmap v1429: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:13 smithi155 ceph-mon[33846]: pgmap v1430: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:14.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:13 smithi043 ceph-mon[30062]: pgmap v1430: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:14.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:13 smithi043 ceph-mon[41699]: pgmap v1430: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:16.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:15 smithi155 ceph-mon[33846]: pgmap v1431: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:16.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:15 smithi043 ceph-mon[30062]: pgmap v1431: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:16.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:15 smithi043 ceph-mon[41699]: pgmap v1431: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:17 smithi155 ceph-mon[33846]: pgmap v1432: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:18.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:17 smithi043 ceph-mon[30062]: pgmap v1432: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:18.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:17 smithi043 ceph-mon[41699]: pgmap v1432: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:18.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:17:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:17:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:17:19.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:17:19 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:17:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:19 smithi155 ceph-mon[33846]: pgmap v1433: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:19 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:17:20.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:19 smithi043 ceph-mon[30062]: pgmap v1433: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:20.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:19 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:17:20.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:19 smithi043 ceph-mon[41699]: pgmap v1433: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:20.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:19 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:17:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:20 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:17:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:20 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:17:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:20 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:17:21.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:20 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:17:21.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:20 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:17:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:20 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:17:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:20 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:17:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:20 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:17:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:20 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:17:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:20 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:17:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:20 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:17:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:20 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:17:22.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:21 smithi155 ceph-mon[33846]: pgmap v1434: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:22.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:21 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:17:22.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:21 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:17:22.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:21 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:17:22.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:21 smithi043 ceph-mon[30062]: pgmap v1434: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:22.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:21 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:17:22.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:21 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:17:22.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:21 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:17:22.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:21 smithi043 ceph-mon[41699]: pgmap v1434: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:22.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:21 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:17:22.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:21 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:17:22.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:21 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:17:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:23 smithi155 ceph-mon[33846]: pgmap v1435: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:17:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:23 smithi043 ceph-mon[30062]: pgmap v1435: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:17:24.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:23 smithi043 ceph-mon[41699]: pgmap v1435: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:17:26.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:25 smithi155 ceph-mon[33846]: pgmap v1436: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:26.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:25 smithi043 ceph-mon[30062]: pgmap v1436: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:26.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:25 smithi043 ceph-mon[41699]: pgmap v1436: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:27 smithi155 ceph-mon[33846]: pgmap v1437: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:28.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:27 smithi043 ceph-mon[30062]: pgmap v1437: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:28.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:17:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:17:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:17:28.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:27 smithi043 ceph-mon[41699]: pgmap v1437: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:29.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:17:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:17:30.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:29 smithi155 ceph-mon[33846]: pgmap v1438: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:30.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:29 smithi043 ceph-mon[30062]: pgmap v1438: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:30.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:29 smithi043 ceph-mon[41699]: pgmap v1438: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:31.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:30 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:17:31.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:30 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:17:31.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:30 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:17:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:31 smithi155 ceph-mon[33846]: pgmap v1439: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:32.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:31 smithi043 ceph-mon[30062]: pgmap v1439: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:31 smithi043 ceph-mon[41699]: pgmap v1439: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:34.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:33 smithi155 ceph-mon[33846]: pgmap v1440: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:34.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:33 smithi043 ceph-mon[30062]: pgmap v1440: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:33 smithi043 ceph-mon[41699]: pgmap v1440: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:35 smithi155 ceph-mon[33846]: pgmap v1441: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:17:36.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:35 smithi043 ceph-mon[30062]: pgmap v1441: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:36.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:17:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:35 smithi043 ceph-mon[41699]: pgmap v1441: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:17:38.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:37 smithi155 ceph-mon[33846]: pgmap v1442: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:38.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:37 smithi043 ceph-mon[30062]: pgmap v1442: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:38.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:17:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:17:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:17:38.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:37 smithi043 ceph-mon[41699]: pgmap v1442: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:39.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:17:39 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:17:40.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:39 smithi155 ceph-mon[33846]: pgmap v1443: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:40.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:39 smithi043 ceph-mon[30062]: pgmap v1443: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:39 smithi043 ceph-mon[41699]: pgmap v1443: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:40 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:17:41.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:40 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:17:41.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:40 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:17:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:41 smithi155 ceph-mon[33846]: pgmap v1444: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:41 smithi043 ceph-mon[30062]: pgmap v1444: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:41 smithi043 ceph-mon[41699]: pgmap v1444: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:44.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:43 smithi155 ceph-mon[33846]: pgmap v1445: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:17:44.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:43 smithi043 ceph-mon[30062]: pgmap v1445: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:17:44.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:43 smithi043 ceph-mon[41699]: pgmap v1445: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:17:46.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:45 smithi155 ceph-mon[33846]: pgmap v1446: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:46.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:45 smithi043 ceph-mon[30062]: pgmap v1446: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:46.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:45 smithi043 ceph-mon[41699]: pgmap v1446: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:47 smithi155 ceph-mon[33846]: pgmap v1447: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:48.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:47 smithi043 ceph-mon[30062]: pgmap v1447: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:48.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:47 smithi043 ceph-mon[41699]: pgmap v1447: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:48.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:17:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:17:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:17:49.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:17:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:17:50.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:49 smithi155 ceph-mon[33846]: pgmap v1448: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:50.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:49 smithi043 ceph-mon[41699]: pgmap v1448: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:49 smithi043 ceph-mon[30062]: pgmap v1448: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:50 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:17:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:50 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:17:51.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:50 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:17:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:50 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:17:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:50 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:17:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:50 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:17:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:51 smithi155 ceph-mon[33846]: pgmap v1449: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:52.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:51 smithi043 ceph-mon[30062]: pgmap v1449: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:51 smithi043 ceph-mon[41699]: pgmap v1449: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:53 smithi155 ceph-mon[33846]: pgmap v1450: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:54.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:53 smithi043 ceph-mon[30062]: pgmap v1450: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:54.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:53 smithi043 ceph-mon[41699]: pgmap v1450: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:55 smithi155 ceph-mon[33846]: pgmap v1451: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:56.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:55 smithi043 ceph-mon[30062]: pgmap v1451: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:56.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:55 smithi043 ceph-mon[41699]: pgmap v1451: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:17:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:57 smithi155 ceph-mon[33846]: pgmap v1452: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:58.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:57 smithi043 ceph-mon[30062]: pgmap v1452: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:58.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:17:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:17:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:17:58.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:57 smithi043 ceph-mon[41699]: pgmap v1452: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:17:59.707 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:17:59 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:18:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:17:59 smithi155 ceph-mon[33846]: pgmap v1453: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:00.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:17:59 smithi043 ceph-mon[30062]: pgmap v1453: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:17:59 smithi043 ceph-mon[41699]: pgmap v1453: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:00 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:18:01.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:00 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:18:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:00 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:18:02.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:01 smithi155 ceph-mon[33846]: pgmap v1454: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:01 smithi043 ceph-mon[30062]: pgmap v1454: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:01 smithi043 ceph-mon[41699]: pgmap v1454: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:03 smithi155 ceph-mon[33846]: pgmap v1455: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:04.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:03 smithi043 ceph-mon[30062]: pgmap v1455: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:04.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:03 smithi043 ceph-mon[41699]: pgmap v1455: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:05 smithi155 ceph-mon[33846]: pgmap v1456: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:05 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:18:06.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:05 smithi043 ceph-mon[30062]: pgmap v1456: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:06.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:05 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:18:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:05 smithi043 ceph-mon[41699]: pgmap v1456: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:05 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:18:07.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:06 smithi043 ceph-mon[30062]: pgmap v1457: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:07.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:06 smithi043 ceph-mon[41699]: pgmap v1457: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:07.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:06 smithi155 ceph-mon[33846]: pgmap v1457: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:08.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:18:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:18:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:18:09.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:09 smithi043 ceph-mon[30062]: pgmap v1458: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:09.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:09 smithi043 ceph-mon[41699]: pgmap v1458: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:09 smithi155 ceph-mon[33846]: pgmap v1458: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:09.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:18:09 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:18:10.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:10 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:18:10.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:10 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:18:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:10 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:18:11.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:11 smithi043 ceph-mon[30062]: pgmap v1459: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:11.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:11 smithi043 ceph-mon[41699]: pgmap v1459: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:11 smithi155 ceph-mon[33846]: pgmap v1459: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:13.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:13 smithi043 ceph-mon[30062]: pgmap v1460: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:18:13.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:13 smithi043 ceph-mon[41699]: pgmap v1460: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:18:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:13 smithi155 ceph-mon[33846]: pgmap v1460: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:18:15.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:15 smithi043 ceph-mon[30062]: pgmap v1461: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:15.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:15 smithi043 ceph-mon[41699]: pgmap v1461: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:15.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:15 smithi155 ceph-mon[33846]: pgmap v1461: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:17.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:17 smithi155 ceph-mon[33846]: pgmap v1462: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:17.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:17 smithi043 ceph-mon[30062]: pgmap v1462: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:17.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:17 smithi043 ceph-mon[41699]: pgmap v1462: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:18.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:18:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:18:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:18:19.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:19 smithi155 ceph-mon[33846]: pgmap v1463: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:19.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:18:19 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:18:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:19 smithi043 ceph-mon[30062]: pgmap v1463: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:19.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:19 smithi043 ceph-mon[41699]: pgmap v1463: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:20.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:20 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:18:20.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:20 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:18:20.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:20 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:18:20.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:20 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:18:20.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:20 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:18:20.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:20 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:18:21.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:21 smithi155 ceph-mon[33846]: pgmap v1464: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:21.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:21 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:18:21.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:21 smithi043 ceph-mon[30062]: pgmap v1464: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:21.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:21 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:18:21.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:21 smithi043 ceph-mon[41699]: pgmap v1464: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:21.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:21 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:18:22.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:22 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:18:22.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:22 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:18:22.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:22 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:18:22.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:22 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:18:22.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:22 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:18:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:22 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:18:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:22 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:18:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:22 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:18:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:22 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:18:22.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:22 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:18:22.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:22 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:18:22.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:22 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:18:22.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:22 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:18:22.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:22 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:18:22.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:22 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:18:23.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:23 smithi155 ceph-mon[33846]: pgmap v1465: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:18:23.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:23 smithi043 ceph-mon[30062]: pgmap v1465: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:18:23.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:23 smithi043 ceph-mon[41699]: pgmap v1465: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:18:25.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:25 smithi155 ceph-mon[33846]: pgmap v1466: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:25.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:25 smithi043 ceph-mon[30062]: pgmap v1466: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:25.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:25 smithi043 ceph-mon[41699]: pgmap v1466: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:27.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:27 smithi155 ceph-mon[33846]: pgmap v1467: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:27.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:27 smithi043 ceph-mon[30062]: pgmap v1467: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:27.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:27 smithi043 ceph-mon[41699]: pgmap v1467: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:28.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:18:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:18:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:18:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:29 smithi155 ceph-mon[33846]: pgmap v1468: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:29.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:18:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:18:29.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:29 smithi043 ceph-mon[30062]: pgmap v1468: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:29.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:29 smithi043 ceph-mon[41699]: pgmap v1468: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:30.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:30 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:18:30.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:30 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:18:30.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:30 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:18:31.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:31 smithi155 ceph-mon[33846]: pgmap v1469: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:31.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:31 smithi043 ceph-mon[30062]: pgmap v1469: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:31.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:31 smithi043 ceph-mon[41699]: pgmap v1469: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:33.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:33 smithi155 ceph-mon[33846]: pgmap v1470: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:18:33.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:33 smithi043 ceph-mon[30062]: pgmap v1470: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:18:33.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:33 smithi043 ceph-mon[41699]: pgmap v1470: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:18:35.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:35 smithi155 ceph-mon[33846]: pgmap v1471: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:35.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:18:35.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:35 smithi043 ceph-mon[30062]: pgmap v1471: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:18:35.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:35 smithi043 ceph-mon[41699]: pgmap v1471: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:35.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:18:37.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:37 smithi155 ceph-mon[33846]: pgmap v1472: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:37.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:37 smithi043 ceph-mon[30062]: pgmap v1472: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:37.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:37 smithi043 ceph-mon[41699]: pgmap v1472: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:38.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:18:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:18:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:18:39.760 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:39 smithi155 ceph-mon[33846]: pgmap v1473: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:39.760 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:18:39 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:18:39.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:39 smithi043 ceph-mon[30062]: pgmap v1473: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:39.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:39 smithi043 ceph-mon[41699]: pgmap v1473: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:40.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:40 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:18:40.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:40 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:18:40.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:40 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:18:41.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:41 smithi155 ceph-mon[33846]: pgmap v1474: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:41.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:41 smithi043 ceph-mon[30062]: pgmap v1474: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:41.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:41 smithi043 ceph-mon[41699]: pgmap v1474: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:43.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:43 smithi155 ceph-mon[33846]: pgmap v1475: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:43.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:43 smithi043 ceph-mon[30062]: pgmap v1475: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:43.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:43 smithi043 ceph-mon[41699]: pgmap v1475: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:45.739 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.739 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.740 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.741 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.741 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.741 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.741 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.741 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.741 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.741 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.741 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.741 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.741 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.741 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.741 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.741 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.741 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.741 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.742 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.742 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.742 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.742 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.742 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.742 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.742 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.742 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.742 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.742 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.742 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.742 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.742 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.743 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.743 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.743 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.743 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.743 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.743 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.743 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.743 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.743 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.743 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.743 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.743 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.743 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.743 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.744 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.744 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.744 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.744 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.744 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.744 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.744 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.744 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.744 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.744 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.744 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.744 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.744 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.744 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.744 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.745 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.745 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.745 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.745 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.745 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.745 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.745 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.745 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.745 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.745 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.745 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.745 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.745 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.745 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.746 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.746 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.746 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.746 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.746 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.746 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.746 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.746 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.746 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.746 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.746 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.746 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.747 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.747 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:18:45.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:45 smithi155 ceph-mon[33846]: pgmap v1476: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:45.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:45 smithi043 ceph-mon[30062]: pgmap v1476: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:45.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:45 smithi043 ceph-mon[41699]: pgmap v1476: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:47.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:47 smithi155 ceph-mon[33846]: pgmap v1477: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:47.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:47 smithi043 ceph-mon[30062]: pgmap v1477: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:47.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:47 smithi043 ceph-mon[41699]: pgmap v1477: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:48.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:18:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:18:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:18:49.801 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:18:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:18:50.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:49 smithi155 ceph-mon[33846]: pgmap v1478: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:50.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:49 smithi043 ceph-mon[30062]: pgmap v1478: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:50.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:49 smithi043 ceph-mon[41699]: pgmap v1478: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:50 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:18:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:50 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:18:51.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:50 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:18:51.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:50 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:18:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:50 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:18:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:50 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:18:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:51 smithi155 ceph-mon[33846]: pgmap v1479: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:52.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:51 smithi043 ceph-mon[30062]: pgmap v1479: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:51 smithi043 ceph-mon[41699]: pgmap v1479: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:53 smithi155 ceph-mon[33846]: pgmap v1480: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:18:54.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:53 smithi043 ceph-mon[30062]: pgmap v1480: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:18:54.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:53 smithi043 ceph-mon[41699]: pgmap v1480: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:18:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:55 smithi155 ceph-mon[33846]: pgmap v1481: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:56.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:55 smithi043 ceph-mon[30062]: pgmap v1481: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:56.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:55 smithi043 ceph-mon[41699]: pgmap v1481: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:18:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:57 smithi155 ceph-mon[33846]: pgmap v1482: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:58.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:57 smithi043 ceph-mon[30062]: pgmap v1482: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:58.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:57 smithi043 ceph-mon[41699]: pgmap v1482: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:18:58.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:18:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:18:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:18:59.819 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:18:59 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:19:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:18:59 smithi155 ceph-mon[33846]: pgmap v1483: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:00.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:18:59 smithi043 ceph-mon[30062]: pgmap v1483: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:18:59 smithi043 ceph-mon[41699]: pgmap v1483: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:00 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:19:01.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:00 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:19:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:00 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:19:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:01 smithi155 ceph-mon[33846]: pgmap v1484: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:02.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:01 smithi043 ceph-mon[30062]: pgmap v1484: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:02.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:01 smithi043 ceph-mon[41699]: pgmap v1484: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:03 smithi155 ceph-mon[33846]: pgmap v1485: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:04.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:03 smithi043 ceph-mon[30062]: pgmap v1485: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:03 smithi043 ceph-mon[41699]: pgmap v1485: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:05 smithi155 ceph-mon[33846]: pgmap v1486: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:05 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:19:06.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:05 smithi043 ceph-mon[30062]: pgmap v1486: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:06.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:05 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:19:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:05 smithi043 ceph-mon[41699]: pgmap v1486: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:05 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:19:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:07 smithi155 ceph-mon[33846]: pgmap v1487: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:08.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:07 smithi043 ceph-mon[30062]: pgmap v1487: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:08.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:19:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:19:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:19:08.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:07 smithi043 ceph-mon[41699]: pgmap v1487: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:09.836 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:19:09 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:19:10.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:09 smithi155 ceph-mon[33846]: pgmap v1488: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:10.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:09 smithi043 ceph-mon[30062]: pgmap v1488: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:10.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:09 smithi043 ceph-mon[41699]: pgmap v1488: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:11.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:10 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:19:11.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:10 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:19:11.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:10 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:19:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:11 smithi155 ceph-mon[33846]: pgmap v1489: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:11 smithi043 ceph-mon[30062]: pgmap v1489: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:12.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:11 smithi043 ceph-mon[41699]: pgmap v1489: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:13 smithi155 ceph-mon[33846]: pgmap v1490: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:14.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:13 smithi043 ceph-mon[30062]: pgmap v1490: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:14.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:13 smithi043 ceph-mon[41699]: pgmap v1490: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:16.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:15 smithi155 ceph-mon[33846]: pgmap v1491: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:16.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:15 smithi043 ceph-mon[30062]: pgmap v1491: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:16.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:15 smithi043 ceph-mon[41699]: pgmap v1491: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:17 smithi155 ceph-mon[33846]: pgmap v1492: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:18.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:17 smithi043 ceph-mon[30062]: pgmap v1492: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:18.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:19:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:19:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:19:18.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:17 smithi043 ceph-mon[41699]: pgmap v1492: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:19.854 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:19:19 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:19:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:19 smithi155 ceph-mon[33846]: pgmap v1493: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:20.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:19 smithi043 ceph-mon[30062]: pgmap v1493: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:20.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:19 smithi043 ceph-mon[41699]: pgmap v1493: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:20 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:19:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:20 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:19:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:20 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:19:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:20 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:19:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:20 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:19:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:20 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:19:22.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:21 smithi155 ceph-mon[33846]: pgmap v1494: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:22.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:21 smithi043 ceph-mon[30062]: pgmap v1494: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:22.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:21 smithi043 ceph-mon[41699]: pgmap v1494: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:23.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:22 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:19:23.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:22 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:19:23.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:22 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:19:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:23 smithi155 ceph-mon[33846]: pgmap v1495: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:23 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:19:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:23 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:19:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:23 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:19:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:23 smithi043 ceph-mon[30062]: pgmap v1495: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:23 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:19:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:23 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:19:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:23 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:19:24.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:23 smithi043 ceph-mon[41699]: pgmap v1495: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:24.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:23 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:19:24.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:23 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:19:24.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:23 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:19:26.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:25 smithi155 ceph-mon[33846]: pgmap v1496: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:26.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:25 smithi043 ceph-mon[30062]: pgmap v1496: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:26.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:25 smithi043 ceph-mon[41699]: pgmap v1496: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:27 smithi155 ceph-mon[33846]: pgmap v1497: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:28.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:27 smithi043 ceph-mon[30062]: pgmap v1497: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:28.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:19:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:19:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:19:28.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:27 smithi043 ceph-mon[41699]: pgmap v1497: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:29.874 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:19:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:19:30.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:29 smithi155 ceph-mon[33846]: pgmap v1498: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:29 smithi043 ceph-mon[30062]: pgmap v1498: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:30.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:29 smithi043 ceph-mon[41699]: pgmap v1498: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:31.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:30 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:19:31.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:30 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:19:31.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:30 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:19:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:31 smithi155 ceph-mon[33846]: pgmap v1499: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:32.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:31 smithi043 ceph-mon[30062]: pgmap v1499: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:31 smithi043 ceph-mon[41699]: pgmap v1499: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:34.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:33 smithi155 ceph-mon[33846]: pgmap v1500: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:34.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:33 smithi043 ceph-mon[30062]: pgmap v1500: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:33 smithi043 ceph-mon[41699]: pgmap v1500: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:35 smithi155 ceph-mon[33846]: pgmap v1501: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:19:36.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:35 smithi043 ceph-mon[30062]: pgmap v1501: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:36.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:19:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:35 smithi043 ceph-mon[41699]: pgmap v1501: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:19:38.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:37 smithi155 ceph-mon[33846]: pgmap v1502: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:38.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:37 smithi043 ceph-mon[30062]: pgmap v1502: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:38.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:19:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:19:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:19:38.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:37 smithi043 ceph-mon[41699]: pgmap v1502: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:39.893 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:19:39 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:19:40.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:39 smithi155 ceph-mon[33846]: pgmap v1503: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:40.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:39 smithi043 ceph-mon[30062]: pgmap v1503: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:39 smithi043 ceph-mon[41699]: pgmap v1503: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:40 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:19:41.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:40 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:19:41.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:40 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:19:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:41 smithi155 ceph-mon[33846]: pgmap v1504: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:42.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:41 smithi043 ceph-mon[30062]: pgmap v1504: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:41 smithi043 ceph-mon[41699]: pgmap v1504: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:44.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:43 smithi155 ceph-mon[33846]: pgmap v1505: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:44.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:43 smithi043 ceph-mon[30062]: pgmap v1505: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:44.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:43 smithi043 ceph-mon[41699]: pgmap v1505: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:46.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:45 smithi155 ceph-mon[33846]: pgmap v1506: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:46.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:45 smithi043 ceph-mon[30062]: pgmap v1506: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:46.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:45 smithi043 ceph-mon[41699]: pgmap v1506: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:47 smithi155 ceph-mon[33846]: pgmap v1507: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:48.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:47 smithi043 ceph-mon[30062]: pgmap v1507: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:48.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:19:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:19:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:19:48.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:47 smithi043 ceph-mon[41699]: pgmap v1507: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:49.911 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:19:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:19:50.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:49 smithi155 ceph-mon[33846]: pgmap v1508: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:50.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:49 smithi043 ceph-mon[30062]: pgmap v1508: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:50.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:49 smithi043 ceph-mon[41699]: pgmap v1508: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:50 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:19:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:50 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:19:51.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:50 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:19:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:50 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:19:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:50 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:19:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:50 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:19:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:51 smithi155 ceph-mon[33846]: pgmap v1509: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:52.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:51 smithi043 ceph-mon[30062]: pgmap v1509: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:51 smithi043 ceph-mon[41699]: pgmap v1509: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:53 smithi155 ceph-mon[33846]: pgmap v1510: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:54.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:53 smithi043 ceph-mon[30062]: pgmap v1510: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:54.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:53 smithi043 ceph-mon[41699]: pgmap v1510: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:55 smithi155 ceph-mon[33846]: pgmap v1511: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:56.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:55 smithi043 ceph-mon[30062]: pgmap v1511: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:56.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:55 smithi043 ceph-mon[41699]: pgmap v1511: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:19:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:57 smithi155 ceph-mon[33846]: pgmap v1512: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:58.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:57 smithi043 ceph-mon[30062]: pgmap v1512: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:58.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:57 smithi043 ceph-mon[41699]: pgmap v1512: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:19:58.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:19:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:19:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:19:59.928 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:19:59 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:20:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:19:59 smithi155 ceph-mon[33846]: pgmap v1513: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:00.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:19:59 smithi043 ceph-mon[30062]: pgmap v1513: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:00.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:19:59 smithi043 ceph-mon[41699]: pgmap v1513: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:00 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:20:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:00 smithi155 ceph-mon[33846]: overall HEALTH_WARN 5 pool(s) do not have an application enabled 2024-03-21T13:20:01.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:00 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:20:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:00 smithi043 ceph-mon[30062]: overall HEALTH_WARN 5 pool(s) do not have an application enabled 2024-03-21T13:20:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:00 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:20:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:00 smithi043 ceph-mon[41699]: overall HEALTH_WARN 5 pool(s) do not have an application enabled 2024-03-21T13:20:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:01 smithi155 ceph-mon[33846]: pgmap v1514: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:02.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:01 smithi043 ceph-mon[30062]: pgmap v1514: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:01 smithi043 ceph-mon[41699]: pgmap v1514: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:03 smithi155 ceph-mon[33846]: pgmap v1515: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:04.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:03 smithi043 ceph-mon[30062]: pgmap v1515: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:04.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:03 smithi043 ceph-mon[41699]: pgmap v1515: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:05 smithi155 ceph-mon[33846]: pgmap v1516: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:05 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:20:06.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:05 smithi043 ceph-mon[30062]: pgmap v1516: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:05 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:20:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:05 smithi043 ceph-mon[41699]: pgmap v1516: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:05 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:20:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:07 smithi155 ceph-mon[33846]: pgmap v1517: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:08.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:07 smithi043 ceph-mon[30062]: pgmap v1517: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:08.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:07 smithi043 ceph-mon[41699]: pgmap v1517: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:08.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:20:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:20:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:20:09.946 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:20:09 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:20:10.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:09 smithi155 ceph-mon[33846]: pgmap v1518: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:10.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:09 smithi043 ceph-mon[30062]: pgmap v1518: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:10.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:09 smithi043 ceph-mon[41699]: pgmap v1518: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:11.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:10 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:20:11.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:10 smithi155 ceph-mon[33846]: pgmap v1519: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:11.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:10 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:20:11.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:10 smithi043 ceph-mon[30062]: pgmap v1519: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:11.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:10 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:20:11.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:10 smithi043 ceph-mon[41699]: pgmap v1519: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:13 smithi155 ceph-mon[33846]: pgmap v1520: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:13.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:13 smithi043 ceph-mon[30062]: pgmap v1520: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:13.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:13 smithi043 ceph-mon[41699]: pgmap v1520: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:15.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:15 smithi155 ceph-mon[33846]: pgmap v1521: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:15.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:15 smithi043 ceph-mon[30062]: pgmap v1521: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:15.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:15 smithi043 ceph-mon[41699]: pgmap v1521: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:17.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:17 smithi155 ceph-mon[33846]: pgmap v1522: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:17.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:17 smithi043 ceph-mon[30062]: pgmap v1522: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:17.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:17 smithi043 ceph-mon[41699]: pgmap v1522: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:18.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:20:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:20:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:20:19.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:19 smithi155 ceph-mon[33846]: pgmap v1523: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:19.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:20:19 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:20:20.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:19 smithi043 ceph-mon[30062]: pgmap v1523: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:20.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:19 smithi043 ceph-mon[41699]: pgmap v1523: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:20 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:20:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:20 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:20:21.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:20 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:20:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:20 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:20:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:20 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:20:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:20 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:20:22.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:21 smithi155 ceph-mon[33846]: pgmap v1524: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:22.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:21 smithi043 ceph-mon[30062]: pgmap v1524: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:22.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:21 smithi043 ceph-mon[41699]: pgmap v1524: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:23 smithi155 ceph-mon[33846]: pgmap v1525: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:20:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:23 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:20:24.211 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:23 smithi043 ceph-mon[30062]: pgmap v1525: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:20:24.211 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:23 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:20:24.211 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:23 smithi043 ceph-mon[41699]: pgmap v1525: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:20:24.211 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:23 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:20:25.752 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.752 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.752 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.752 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.752 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.752 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.752 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.752 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.752 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.752 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.752 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.752 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.753 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.753 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.753 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.753 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.753 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.753 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.753 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.753 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.753 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.753 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.753 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.753 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.753 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.753 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.753 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.753 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.754 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.754 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.754 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.754 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.754 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.754 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.754 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.754 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.754 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.754 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.754 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.754 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.754 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.754 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.754 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.755 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.755 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.755 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.755 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.755 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.755 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.755 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.755 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.755 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.755 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.755 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.755 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.755 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.755 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.755 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.755 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.755 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.755 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.756 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.756 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.756 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.756 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.756 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.756 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.756 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.756 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.756 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.756 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.756 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.756 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.756 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.756 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.756 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.756 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.757 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.757 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.757 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.757 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.757 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.757 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.757 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.757 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.757 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.757 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.757 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.757 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.757 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.757 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.757 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.757 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.757 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.758 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.758 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.758 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.758 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.758 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.758 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:20:25.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:25 smithi043 ceph-mon[30062]: pgmap v1526: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:25.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:25 smithi043 ceph-mon[41699]: pgmap v1526: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:26.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:25 smithi155 ceph-mon[33846]: pgmap v1526: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:27 smithi155 ceph-mon[33846]: pgmap v1527: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:20:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:27 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:20:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:27 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:20:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:27 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:20:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:27 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:20:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:27 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:20:28.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:20:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:20:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:20:28.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:27 smithi043 ceph-mon[30062]: pgmap v1527: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:20:28.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:27 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:20:28.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:27 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:20:28.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:27 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:20:28.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:27 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:20:28.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:27 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:20:28.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:27 smithi043 ceph-mon[41699]: pgmap v1527: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:20:28.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:27 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:20:28.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:27 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:20:28.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:27 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:20:28.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:27 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:20:28.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:27 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:20:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:29 smithi155 ceph-mon[33846]: pgmap v1528: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:29.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:20:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:20:30.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:29 smithi043 ceph-mon[30062]: pgmap v1528: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:30.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:29 smithi043 ceph-mon[41699]: pgmap v1528: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:31.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:30 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:20:31.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:30 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:20:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:30 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:20:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:31 smithi155 ceph-mon[33846]: pgmap v1529: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:20:32.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:31 smithi043 ceph-mon[30062]: pgmap v1529: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:20:32.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:31 smithi043 ceph-mon[41699]: pgmap v1529: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:20:34.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:33 smithi155 ceph-mon[33846]: pgmap v1530: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:20:34.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:33 smithi043 ceph-mon[30062]: pgmap v1530: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:20:34.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:33 smithi043 ceph-mon[41699]: pgmap v1530: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:20:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:35 smithi155 ceph-mon[33846]: pgmap v1531: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:20:36.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:35 smithi043 ceph-mon[30062]: pgmap v1531: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:36.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:20:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:35 smithi043 ceph-mon[41699]: pgmap v1531: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:20:38.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:37 smithi155 ceph-mon[33846]: pgmap v1532: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:38.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:37 smithi043 ceph-mon[30062]: pgmap v1532: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:38.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:20:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:20:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:20:38.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:37 smithi043 ceph-mon[41699]: pgmap v1532: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:39.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:39 smithi155 ceph-mon[33846]: pgmap v1533: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:39.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:20:39 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:20:40.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:39 smithi043 ceph-mon[30062]: pgmap v1533: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:39 smithi043 ceph-mon[41699]: pgmap v1533: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:40 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:20:41.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:40 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:20:41.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:40 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:20:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:41 smithi155 ceph-mon[33846]: pgmap v1534: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:42.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:41 smithi043 ceph-mon[30062]: pgmap v1534: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:41 smithi043 ceph-mon[41699]: pgmap v1534: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:44.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:43 smithi155 ceph-mon[33846]: pgmap v1535: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:20:44.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:43 smithi043 ceph-mon[30062]: pgmap v1535: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:20:44.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:43 smithi043 ceph-mon[41699]: pgmap v1535: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:20:46.134 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:45 smithi155 ceph-mon[33846]: pgmap v1536: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:46.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:45 smithi043 ceph-mon[30062]: pgmap v1536: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:46.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:45 smithi043 ceph-mon[41699]: pgmap v1536: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:47 smithi155 ceph-mon[33846]: pgmap v1537: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:48.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:47 smithi043 ceph-mon[30062]: pgmap v1537: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:48.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:47 smithi043 ceph-mon[41699]: pgmap v1537: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:48.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:20:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:20:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:20:49.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:49 smithi155 ceph-mon[33846]: pgmap v1538: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:49.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:20:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:20:50.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:49 smithi043 ceph-mon[30062]: pgmap v1538: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:50.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:49 smithi043 ceph-mon[41699]: pgmap v1538: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:50 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:20:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:50 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:20:51.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:50 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:20:51.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:50 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:20:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:50 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:20:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:50 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:20:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:51 smithi155 ceph-mon[33846]: pgmap v1539: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:52.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:51 smithi043 ceph-mon[30062]: pgmap v1539: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:52.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:51 smithi043 ceph-mon[41699]: pgmap v1539: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:53 smithi155 ceph-mon[33846]: pgmap v1540: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:20:54.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:53 smithi043 ceph-mon[30062]: pgmap v1540: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:20:54.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:53 smithi043 ceph-mon[41699]: pgmap v1540: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:20:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:55 smithi155 ceph-mon[33846]: pgmap v1541: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:56.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:55 smithi043 ceph-mon[30062]: pgmap v1541: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:56.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:55 smithi043 ceph-mon[41699]: pgmap v1541: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:20:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:57 smithi155 ceph-mon[33846]: pgmap v1542: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:58.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:57 smithi043 ceph-mon[30062]: pgmap v1542: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:58.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:57 smithi043 ceph-mon[41699]: pgmap v1542: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:20:58.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:20:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:20:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:20:59.889 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:20:59 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:21:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:20:59 smithi155 ceph-mon[33846]: pgmap v1543: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:00.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:20:59 smithi043 ceph-mon[30062]: pgmap v1543: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:00.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:20:59 smithi043 ceph-mon[41699]: pgmap v1543: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:00 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:21:01.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:00 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:21:01.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:00 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:21:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:01 smithi155 ceph-mon[33846]: pgmap v1544: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:01 smithi043 ceph-mon[30062]: pgmap v1544: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:01 smithi043 ceph-mon[41699]: pgmap v1544: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:03 smithi155 ceph-mon[33846]: pgmap v1545: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:21:04.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:03 smithi043 ceph-mon[30062]: pgmap v1545: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:21:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:03 smithi043 ceph-mon[41699]: pgmap v1545: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:21:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:05 smithi155 ceph-mon[33846]: pgmap v1546: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:05 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:21:06.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:05 smithi043 ceph-mon[30062]: pgmap v1546: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:06.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:05 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:21:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:05 smithi043 ceph-mon[41699]: pgmap v1546: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:05 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:21:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:07 smithi155 ceph-mon[33846]: pgmap v1547: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:08.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:07 smithi043 ceph-mon[30062]: pgmap v1547: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:08.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:07 smithi043 ceph-mon[41699]: pgmap v1547: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:08.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:21:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:21:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:21:09.907 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:21:09 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:21:10.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:09 smithi155 ceph-mon[33846]: pgmap v1548: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:10.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:09 smithi043 ceph-mon[30062]: pgmap v1548: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:10.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:09 smithi043 ceph-mon[41699]: pgmap v1548: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:11.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:10 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:21:11.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:10 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:21:11.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:10 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:21:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:11 smithi155 ceph-mon[33846]: pgmap v1549: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:12.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:11 smithi043 ceph-mon[30062]: pgmap v1549: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:12.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:11 smithi043 ceph-mon[41699]: pgmap v1549: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:13 smithi155 ceph-mon[33846]: pgmap v1550: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:14.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:13 smithi043 ceph-mon[30062]: pgmap v1550: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:14.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:13 smithi043 ceph-mon[41699]: pgmap v1550: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:16.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:15 smithi155 ceph-mon[33846]: pgmap v1551: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:16.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:15 smithi043 ceph-mon[30062]: pgmap v1551: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:16.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:15 smithi043 ceph-mon[41699]: pgmap v1551: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:17 smithi155 ceph-mon[33846]: pgmap v1552: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:18.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:17 smithi043 ceph-mon[30062]: pgmap v1552: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:18.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:21:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:21:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:21:18.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:17 smithi043 ceph-mon[41699]: pgmap v1552: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:19.929 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:21:19 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:21:20.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:19 smithi155 ceph-mon[33846]: pgmap v1553: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:20.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:19 smithi043 ceph-mon[41699]: pgmap v1553: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:20.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:19 smithi043 ceph-mon[30062]: pgmap v1553: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:20 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:21:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:20 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:21:21.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:20 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:21:21.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:20 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:21:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:20 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:21:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:20 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:21:22.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:21 smithi155 ceph-mon[33846]: pgmap v1554: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:22.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:21 smithi043 ceph-mon[30062]: pgmap v1554: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:22.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:21 smithi043 ceph-mon[41699]: pgmap v1554: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:23 smithi155 ceph-mon[33846]: pgmap v1555: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:24.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:23 smithi043 ceph-mon[30062]: pgmap v1555: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:24.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:23 smithi043 ceph-mon[41699]: pgmap v1555: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:26.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:25 smithi155 ceph-mon[33846]: pgmap v1556: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:26.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:25 smithi043 ceph-mon[30062]: pgmap v1556: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:26.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:25 smithi043 ceph-mon[41699]: pgmap v1556: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:27 smithi155 ceph-mon[33846]: pgmap v1557: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:27 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:21:28.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:21:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:21:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:21:28.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:27 smithi043 ceph-mon[30062]: pgmap v1557: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:28.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:27 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:21:28.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:27 smithi043 ceph-mon[41699]: pgmap v1557: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:28.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:27 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:21:29.947 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:21:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:21:30.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:29 smithi155 ceph-mon[33846]: pgmap v1558: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:30.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:29 smithi043 ceph-mon[30062]: pgmap v1558: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:30.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:29 smithi043 ceph-mon[41699]: pgmap v1558: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:31.187 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:30 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:21:31.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:30 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:21:31.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:30 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:21:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:31 smithi155 ceph-mon[33846]: pgmap v1559: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:31 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:21:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:31 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:21:32.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:31 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:21:32.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:31 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:21:32.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:31 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:21:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:31 smithi043 ceph-mon[30062]: pgmap v1559: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:31 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:21:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:31 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:21:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:31 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:21:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:31 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:21:32.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:31 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:21:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:31 smithi043 ceph-mon[41699]: pgmap v1559: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:31 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:21:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:31 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:21:32.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:31 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:21:32.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:31 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:21:32.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:31 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:21:34.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:33 smithi043 ceph-mon[30062]: pgmap v1560: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:21:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:33 smithi043 ceph-mon[41699]: pgmap v1560: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:21:34.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:33 smithi155 ceph-mon[33846]: pgmap v1560: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:21:36.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:35 smithi043 ceph-mon[30062]: pgmap v1561: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:36.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:21:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:35 smithi043 ceph-mon[41699]: pgmap v1561: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:21:36.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:35 smithi155 ceph-mon[33846]: pgmap v1561: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:36.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:21:38.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:37 smithi043 ceph-mon[30062]: pgmap v1562: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:38.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:21:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:21:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:21:38.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:37 smithi043 ceph-mon[41699]: pgmap v1562: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:38.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:37 smithi155 ceph-mon[33846]: pgmap v1562: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:39.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:38 smithi043 ceph-mon[30062]: pgmap v1563: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:39.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:38 smithi043 ceph-mon[41699]: pgmap v1563: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:39.358 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:38 smithi155 ceph-mon[33846]: pgmap v1563: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:21:39.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:21:39 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:21:40.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:39 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:21:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:39 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:21:40.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:39 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:21:41.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:40 smithi043 ceph-mon[30062]: pgmap v1564: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:41.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:40 smithi043 ceph-mon[41699]: pgmap v1564: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:41.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:40 smithi155 ceph-mon[33846]: pgmap v1564: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:21:43.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:43 smithi155 ceph-mon[33846]: pgmap v1565: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:21:43.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:43 smithi043 ceph-mon[30062]: pgmap v1565: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:21:43.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:43 smithi043 ceph-mon[41699]: pgmap v1565: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:21:45.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:45 smithi155 ceph-mon[33846]: pgmap v1566: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:21:45.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:45 smithi043 ceph-mon[30062]: pgmap v1566: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:21:45.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:45 smithi043 ceph-mon[41699]: pgmap v1566: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:21:47.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:47 smithi155 ceph-mon[33846]: pgmap v1567: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:21:47.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:47 smithi043 ceph-mon[30062]: pgmap v1567: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:21:47.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:47 smithi043 ceph-mon[41699]: pgmap v1567: 292 pgs: 292 active+clean; 582 KiB data, 820 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-21T13:21:48.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:21:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:21:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:21:49.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:49 smithi155 ceph-mon[33846]: pgmap v1568: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:21:49.958 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:21:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:21:49.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:49 smithi043 ceph-mon[30062]: pgmap v1568: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:21:49.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:49 smithi043 ceph-mon[41699]: pgmap v1568: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:21:50.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:50 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:21:50.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:50 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:21:50.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:50 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:21:50.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:50 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:21:50.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:50 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:21:50.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:50 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:21:51.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:51 smithi155 ceph-mon[33846]: pgmap v1569: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:21:51.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:51 smithi043 ceph-mon[30062]: pgmap v1569: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:21:51.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:51 smithi043 ceph-mon[41699]: pgmap v1569: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:21:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:53 smithi155 ceph-mon[33846]: pgmap v1570: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:21:54.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:53 smithi043 ceph-mon[30062]: pgmap v1570: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:21:54.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:53 smithi043 ceph-mon[41699]: pgmap v1570: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:21:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:55 smithi155 ceph-mon[33846]: pgmap v1571: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:21:56.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:55 smithi043 ceph-mon[30062]: pgmap v1571: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:21:56.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:55 smithi043 ceph-mon[41699]: pgmap v1571: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:21:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:57 smithi155 ceph-mon[33846]: pgmap v1572: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 59 op/s 2024-03-21T13:21:58.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:57 smithi043 ceph-mon[30062]: pgmap v1572: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 59 op/s 2024-03-21T13:21:58.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:21:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:21:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:21:58.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:57 smithi043 ceph-mon[41699]: pgmap v1572: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 59 op/s 2024-03-21T13:21:59.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:21:59 smithi155 ceph-mon[33846]: pgmap v1573: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:21:59.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:21:59 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:22:00.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:21:59 smithi043 ceph-mon[30062]: pgmap v1573: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:22:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:21:59 smithi043 ceph-mon[41699]: pgmap v1573: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-21T13:22:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:00 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:22:01.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:00 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:22:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:00 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:22:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:01 smithi155 ceph-mon[33846]: pgmap v1574: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:02.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:01 smithi043 ceph-mon[30062]: pgmap v1574: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:01 smithi043 ceph-mon[41699]: pgmap v1574: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:03 smithi155 ceph-mon[33846]: pgmap v1575: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:04.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:03 smithi043 ceph-mon[30062]: pgmap v1575: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:04.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:03 smithi043 ceph-mon[41699]: pgmap v1575: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:04.763 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.763 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.763 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.763 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.763 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.763 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.764 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.764 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.764 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.764 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.764 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.764 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.764 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.764 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.764 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.764 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.764 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.764 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.764 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.764 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.764 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.765 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.765 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.765 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.765 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.765 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.765 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.765 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.765 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.765 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.765 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.765 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.765 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.765 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.765 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.765 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.765 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.766 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.766 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.766 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.766 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.766 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.766 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.766 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.766 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.766 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.766 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.766 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.766 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.766 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.766 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.766 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.767 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.767 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.767 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.767 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.767 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.767 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.767 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.767 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.767 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.767 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.767 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.767 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.767 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.767 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.767 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.767 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.768 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.768 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.768 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.768 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.768 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.768 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.768 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.768 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.768 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.768 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.768 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.768 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.768 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.768 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.769 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.769 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.769 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.769 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.769 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.769 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.769 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.769 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.769 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.769 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.769 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.769 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.769 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.769 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.769 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.770 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:04.770 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:22:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:05 smithi155 ceph-mon[33846]: pgmap v1576: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:05 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:22:06.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:05 smithi043 ceph-mon[30062]: pgmap v1576: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:06.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:05 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:22:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:05 smithi043 ceph-mon[41699]: pgmap v1576: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:05 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:22:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:07 smithi155 ceph-mon[33846]: pgmap v1577: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:08.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:07 smithi043 ceph-mon[30062]: pgmap v1577: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:08.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:07 smithi043 ceph-mon[41699]: pgmap v1577: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:08.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:22:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:22:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:22:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:09 smithi155 ceph-mon[33846]: pgmap v1578: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:09.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:22:09 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:22:10.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:09 smithi043 ceph-mon[30062]: pgmap v1578: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:10.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:09 smithi043 ceph-mon[41699]: pgmap v1578: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:11.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:10 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:22:11.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:10 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:22:11.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:10 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:22:12.176 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:11 smithi155 ceph-mon[33846]: pgmap v1579: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:12.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:11 smithi043 ceph-mon[30062]: pgmap v1579: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:12.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:11 smithi043 ceph-mon[41699]: pgmap v1579: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:14.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:13 smithi155 ceph-mon[33846]: pgmap v1580: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:22:14.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:13 smithi043 ceph-mon[30062]: pgmap v1580: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:22:14.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:13 smithi043 ceph-mon[41699]: pgmap v1580: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:22:16.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:15 smithi155 ceph-mon[33846]: pgmap v1581: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:16.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:15 smithi043 ceph-mon[30062]: pgmap v1581: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:16.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:15 smithi043 ceph-mon[41699]: pgmap v1581: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:17 smithi155 ceph-mon[33846]: pgmap v1582: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:18.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:17 smithi043 ceph-mon[30062]: pgmap v1582: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:18.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:22:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:22:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:22:18.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:17 smithi043 ceph-mon[41699]: pgmap v1582: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:19.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:19 smithi155 ceph-mon[33846]: pgmap v1583: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:19.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:22:19 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:22:20.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:19 smithi043 ceph-mon[30062]: pgmap v1583: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:20.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:19 smithi043 ceph-mon[41699]: pgmap v1583: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:20 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:22:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:20 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:22:21.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:20 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:22:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:20 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:22:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:20 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:22:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:20 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:22:21.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:21 smithi155 ceph-mon[33846]: pgmap v1584: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:22.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:21 smithi043 ceph-mon[30062]: pgmap v1584: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:22.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:21 smithi043 ceph-mon[41699]: pgmap v1584: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:23 smithi155 ceph-mon[33846]: pgmap v1585: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:24.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:23 smithi043 ceph-mon[30062]: pgmap v1585: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:24.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:23 smithi043 ceph-mon[41699]: pgmap v1585: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:26.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:25 smithi155 ceph-mon[33846]: pgmap v1586: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:26.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:25 smithi043 ceph-mon[30062]: pgmap v1586: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:26.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:25 smithi043 ceph-mon[41699]: pgmap v1586: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:27 smithi155 ceph-mon[33846]: pgmap v1587: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:28.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:27 smithi043 ceph-mon[30062]: pgmap v1587: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:28.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:27 smithi043 ceph-mon[41699]: pgmap v1587: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:28.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:22:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:22:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:22:29.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:29 smithi155 ceph-mon[33846]: pgmap v1588: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:29.958 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:22:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:22:30.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:29 smithi043 ceph-mon[30062]: pgmap v1588: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:30.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:29 smithi043 ceph-mon[41699]: pgmap v1588: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:31.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:30 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:22:31.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:30 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:22:31.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:30 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:22:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:31 smithi155 ceph-mon[33846]: pgmap v1589: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:31 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:22:32.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:31 smithi043 ceph-mon[30062]: pgmap v1589: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:32.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:31 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:22:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:31 smithi043 ceph-mon[41699]: pgmap v1589: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:31 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:22:34.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:33 smithi155 ceph-mon[33846]: pgmap v1590: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:22:34.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:33 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:22:34.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:33 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:22:34.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:33 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:34.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:33 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:22:34.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:33 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:22:34.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:33 smithi043 ceph-mon[30062]: pgmap v1590: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:22:34.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:33 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:22:34.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:33 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:22:34.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:33 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:34.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:33 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:22:34.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:33 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:22:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:33 smithi043 ceph-mon[41699]: pgmap v1590: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:22:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:33 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:22:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:33 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:22:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:33 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:22:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:33 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:22:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:33 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:22:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:35 smithi155 ceph-mon[33846]: pgmap v1591: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:22:36.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:35 smithi043 ceph-mon[30062]: pgmap v1591: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:36.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:22:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:35 smithi043 ceph-mon[41699]: pgmap v1591: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:22:38.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:37 smithi155 ceph-mon[33846]: pgmap v1592: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:38.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:37 smithi043 ceph-mon[30062]: pgmap v1592: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:38.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:22:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:22:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:22:38.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:37 smithi043 ceph-mon[41699]: pgmap v1592: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:39.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:39 smithi155 ceph-mon[33846]: pgmap v1593: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:39.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:22:39 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:22:40.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:39 smithi043 ceph-mon[30062]: pgmap v1593: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:39 smithi043 ceph-mon[41699]: pgmap v1593: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:40 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:22:41.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:40 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:22:41.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:40 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:22:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:41 smithi155 ceph-mon[33846]: pgmap v1594: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:42.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:41 smithi043 ceph-mon[30062]: pgmap v1594: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:42.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:41 smithi043 ceph-mon[41699]: pgmap v1594: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:44.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:43 smithi155 ceph-mon[33846]: pgmap v1595: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:22:44.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:43 smithi043 ceph-mon[30062]: pgmap v1595: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:22:44.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:43 smithi043 ceph-mon[41699]: pgmap v1595: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:22:46.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:45 smithi155 ceph-mon[33846]: pgmap v1596: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:46.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:45 smithi043 ceph-mon[30062]: pgmap v1596: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:46.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:45 smithi043 ceph-mon[41699]: pgmap v1596: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:47 smithi155 ceph-mon[33846]: pgmap v1597: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:48.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:47 smithi043 ceph-mon[30062]: pgmap v1597: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:48.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:22:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:22:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:22:48.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:47 smithi043 ceph-mon[41699]: pgmap v1597: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:50.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:49 smithi155 ceph-mon[33846]: pgmap v1598: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:50.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:22:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:22:50.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:49 smithi043 ceph-mon[30062]: pgmap v1598: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:50.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:49 smithi043 ceph-mon[41699]: pgmap v1598: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:50 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:22:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:50 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:22:51.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:50 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:22:51.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:50 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:22:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:50 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:22:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:50 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:22:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:51 smithi155 ceph-mon[33846]: pgmap v1599: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:51 smithi043 ceph-mon[30062]: pgmap v1599: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:51 smithi043 ceph-mon[41699]: pgmap v1599: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:53 smithi155 ceph-mon[33846]: pgmap v1600: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:53 smithi043 ceph-mon[30062]: pgmap v1600: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:54.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:53 smithi043 ceph-mon[41699]: pgmap v1600: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:55 smithi155 ceph-mon[33846]: pgmap v1601: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:56.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:55 smithi043 ceph-mon[30062]: pgmap v1601: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:56.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:55 smithi043 ceph-mon[41699]: pgmap v1601: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:22:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:57 smithi155 ceph-mon[33846]: pgmap v1602: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:58.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:57 smithi043 ceph-mon[30062]: pgmap v1602: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:22:58.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:22:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:22:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:22:58.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:57 smithi043 ceph-mon[41699]: pgmap v1602: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:22:59 smithi155 ceph-mon[33846]: pgmap v1603: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:00.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:22:59 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:23:00.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:22:59 smithi043 ceph-mon[30062]: pgmap v1603: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:00.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:22:59 smithi043 ceph-mon[41699]: pgmap v1603: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:00 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:23:01.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:00 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:23:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:00 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:23:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:01 smithi155 ceph-mon[33846]: pgmap v1604: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:02.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:01 smithi043 ceph-mon[30062]: pgmap v1604: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:01 smithi043 ceph-mon[41699]: pgmap v1604: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:03 smithi155 ceph-mon[33846]: pgmap v1605: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:04.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:03 smithi043 ceph-mon[30062]: pgmap v1605: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:03 smithi043 ceph-mon[41699]: pgmap v1605: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:05 smithi155 ceph-mon[33846]: pgmap v1606: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:05 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:23:06.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:05 smithi043 ceph-mon[30062]: pgmap v1606: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:05 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:23:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:05 smithi043 ceph-mon[41699]: pgmap v1606: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:05 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:23:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:07 smithi155 ceph-mon[33846]: pgmap v1607: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:08.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:07 smithi043 ceph-mon[30062]: pgmap v1607: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:08.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:07 smithi043 ceph-mon[41699]: pgmap v1607: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:08.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:23:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:23:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:23:10.009 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:09 smithi155 ceph-mon[33846]: pgmap v1608: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:10.010 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:23:09 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:23:10.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:09 smithi043 ceph-mon[30062]: pgmap v1608: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:10.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:09 smithi043 ceph-mon[41699]: pgmap v1608: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:11.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:10 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:23:11.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:10 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:23:11.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:10 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:23:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:11 smithi155 ceph-mon[33846]: pgmap v1609: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:12.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:11 smithi043 ceph-mon[30062]: pgmap v1609: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:12.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:11 smithi043 ceph-mon[41699]: pgmap v1609: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:13 smithi155 ceph-mon[33846]: pgmap v1610: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:23:14.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:13 smithi043 ceph-mon[30062]: pgmap v1610: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:23:14.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:13 smithi043 ceph-mon[41699]: pgmap v1610: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:23:16.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:15 smithi155 ceph-mon[33846]: pgmap v1611: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:16.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:15 smithi043 ceph-mon[30062]: pgmap v1611: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:16.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:15 smithi043 ceph-mon[41699]: pgmap v1611: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:17.457 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 13:23:17 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=cleanup t=2024-03-21T13:23:17.153806448Z level=info msg="Completed cleanup jobs" duration=7.097379ms 2024-03-21T13:23:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:17 smithi155 ceph-mon[33846]: pgmap v1612: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:18.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:17 smithi043 ceph-mon[30062]: pgmap v1612: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:18.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:23:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:23:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:23:18.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:17 smithi043 ceph-mon[41699]: pgmap v1612: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:19 smithi155 ceph-mon[33846]: pgmap v1613: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:20.208 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:23:19 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:23:20.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:19 smithi043 ceph-mon[30062]: pgmap v1613: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:20.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:19 smithi043 ceph-mon[41699]: pgmap v1613: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:20 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:23:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:20 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:23:21.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:20 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:23:21.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:20 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:23:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:20 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:23:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:20 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:23:22.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:21 smithi155 ceph-mon[33846]: pgmap v1614: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:22.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:21 smithi043 ceph-mon[30062]: pgmap v1614: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:22.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:21 smithi043 ceph-mon[41699]: pgmap v1614: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:23 smithi155 ceph-mon[33846]: pgmap v1615: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:24.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:23 smithi043 ceph-mon[30062]: pgmap v1615: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:24.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:23 smithi043 ceph-mon[41699]: pgmap v1615: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:26.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:25 smithi155 ceph-mon[33846]: pgmap v1616: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:26.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:25 smithi043 ceph-mon[30062]: pgmap v1616: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:26.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:25 smithi043 ceph-mon[41699]: pgmap v1616: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:27 smithi155 ceph-mon[33846]: pgmap v1617: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:28.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:27 smithi043 ceph-mon[30062]: pgmap v1617: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:28.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:23:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:23:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:23:28.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:27 smithi043 ceph-mon[41699]: pgmap v1617: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:30.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:29 smithi155 ceph-mon[33846]: pgmap v1618: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:30.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:23:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:23:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:29 smithi043 ceph-mon[30062]: pgmap v1618: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:30.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:29 smithi043 ceph-mon[41699]: pgmap v1618: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:31.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:30 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:23:31.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:30 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:23:31.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:30 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:23:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:31 smithi155 ceph-mon[33846]: pgmap v1619: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:32.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:31 smithi043 ceph-mon[30062]: pgmap v1619: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:31 smithi043 ceph-mon[41699]: pgmap v1619: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:34.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:33 smithi155 ceph-mon[33846]: pgmap v1620: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:23:34.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:33 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:23:34.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:33 smithi043 ceph-mon[30062]: pgmap v1620: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:23:34.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:33 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:23:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:33 smithi043 ceph-mon[41699]: pgmap v1620: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:23:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:33 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:23:35.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:35 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:23:35.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:35 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:23:35.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:35.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:23:35.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:35 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:23:35.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:35 smithi155 ceph-mon[33846]: pgmap v1621: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:35.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:35 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:23:35.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:35 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:23:35.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:35.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:23:35.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:35 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:23:35.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:35 smithi043 ceph-mon[30062]: pgmap v1621: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:35.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:35 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:23:35.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:35 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:23:35.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:23:35.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:23:35.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:35 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:23:35.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:35 smithi043 ceph-mon[41699]: pgmap v1621: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:36.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:36 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:23:36.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:36 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:23:36.486 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:36 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:23:37.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:37 smithi155 ceph-mon[33846]: pgmap v1622: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:37.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:37 smithi043 ceph-mon[30062]: pgmap v1622: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:37.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:37 smithi043 ceph-mon[41699]: pgmap v1622: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:38.384 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:23:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:23:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:23:39.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:39 smithi155 ceph-mon[33846]: pgmap v1623: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:39.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:23:39 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:23:39.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:39 smithi043 ceph-mon[30062]: pgmap v1623: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:39.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:39 smithi043 ceph-mon[41699]: pgmap v1623: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:40.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:40 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:23:40.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:40 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:23:40.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:40 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:23:41.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:41 smithi155 ceph-mon[33846]: pgmap v1624: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:41.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:41 smithi043 ceph-mon[30062]: pgmap v1624: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:41.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:41 smithi043 ceph-mon[41699]: pgmap v1624: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:43.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:43 smithi155 ceph-mon[33846]: pgmap v1625: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:23:43.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:43 smithi043 ceph-mon[30062]: pgmap v1625: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:23:43.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:43 smithi043 ceph-mon[41699]: pgmap v1625: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:23:44.776 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.776 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.776 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.776 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.776 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.777 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.777 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.777 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.777 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.777 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.777 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.778 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.778 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.778 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.778 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.778 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.778 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.778 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.779 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.779 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.779 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.779 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.780 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.780 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.780 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.780 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.780 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.781 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.781 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.781 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.781 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.781 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.781 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.782 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.782 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.782 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.782 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.782 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.782 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.783 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.783 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.783 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.783 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.783 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.783 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.783 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.784 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.784 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.784 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.784 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.784 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.784 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.785 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.785 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.785 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.785 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.785 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.785 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.786 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.786 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.786 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.786 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.786 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.786 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.786 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.787 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.787 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.787 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.787 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.788 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.788 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.788 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.788 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.788 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.788 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.789 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.789 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.789 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.789 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.790 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.790 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.790 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.790 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.790 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.791 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.791 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.791 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.791 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.791 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.791 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.791 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.792 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.792 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.792 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.792 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.792 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.792 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.792 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.793 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:44.793 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:23:45.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:45 smithi155 ceph-mon[33846]: pgmap v1626: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:45.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:45 smithi043 ceph-mon[30062]: pgmap v1626: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:45.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:45 smithi043 ceph-mon[41699]: pgmap v1626: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:47.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:47 smithi155 ceph-mon[33846]: pgmap v1627: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:47.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:47 smithi043 ceph-mon[30062]: pgmap v1627: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:47.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:47 smithi043 ceph-mon[41699]: pgmap v1627: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:48.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:23:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:23:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:23:49.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:49 smithi155 ceph-mon[33846]: pgmap v1628: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:49.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:23:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:23:49.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:49 smithi043 ceph-mon[30062]: pgmap v1628: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:49.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:49 smithi043 ceph-mon[41699]: pgmap v1628: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:50.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:50 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:23:50.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:50 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:23:50.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:50 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:23:51.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:51 smithi155 ceph-mon[33846]: pgmap v1629: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:51.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:51 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:23:51.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:51 smithi043 ceph-mon[30062]: pgmap v1629: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:51.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:51 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:23:51.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:51 smithi043 ceph-mon[41699]: pgmap v1629: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:51.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:51 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:23:53.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:53 smithi155 ceph-mon[33846]: pgmap v1630: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:53.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:53 smithi043 ceph-mon[30062]: pgmap v1630: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:53.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:53 smithi043 ceph-mon[41699]: pgmap v1630: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:55.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:55 smithi155 ceph-mon[33846]: pgmap v1631: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:55.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:55 smithi043 ceph-mon[30062]: pgmap v1631: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:55.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:55 smithi043 ceph-mon[41699]: pgmap v1631: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:57.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:57 smithi155 ceph-mon[33846]: pgmap v1632: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:57.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:57 smithi043 ceph-mon[30062]: pgmap v1632: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:57.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:57 smithi043 ceph-mon[41699]: pgmap v1632: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:23:58.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:23:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:23:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:23:59.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:23:59 smithi155 ceph-mon[33846]: pgmap v1633: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:59.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:23:59 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:23:59.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:23:59 smithi043 ceph-mon[30062]: pgmap v1633: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:23:59.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:23:59 smithi043 ceph-mon[41699]: pgmap v1633: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:00.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:00 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:24:00.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:00 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:24:00.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:00 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:24:01.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:01 smithi155 ceph-mon[33846]: pgmap v1634: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:01.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:01 smithi043 ceph-mon[30062]: pgmap v1634: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:01.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:01 smithi043 ceph-mon[41699]: pgmap v1634: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:03.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:03 smithi155 ceph-mon[33846]: pgmap v1635: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:03.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:03 smithi043 ceph-mon[30062]: pgmap v1635: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:03.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:03 smithi043 ceph-mon[41699]: pgmap v1635: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:05.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:05 smithi155 ceph-mon[33846]: pgmap v1636: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:05.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:05 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:24:05.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:05 smithi043 ceph-mon[30062]: pgmap v1636: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:05.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:05 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:24:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:05 smithi043 ceph-mon[41699]: pgmap v1636: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:05 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:24:07.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:07 smithi155 ceph-mon[33846]: pgmap v1637: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:07.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:07 smithi043 ceph-mon[30062]: pgmap v1637: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:07.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:07 smithi043 ceph-mon[41699]: pgmap v1637: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:08.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:24:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:24:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:24:09.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:09 smithi155 ceph-mon[33846]: pgmap v1638: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:09.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:24:09 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:24:09.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:09 smithi043 ceph-mon[30062]: pgmap v1638: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:09.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:09 smithi043 ceph-mon[41699]: pgmap v1638: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:10.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:10 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:24:10.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:10 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:24:10.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:10 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:24:11.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:11 smithi155 ceph-mon[33846]: pgmap v1639: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:11.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:11 smithi043 ceph-mon[30062]: pgmap v1639: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:11.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:11 smithi043 ceph-mon[41699]: pgmap v1639: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:13 smithi155 ceph-mon[33846]: pgmap v1640: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:24:13.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:13 smithi043 ceph-mon[30062]: pgmap v1640: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:24:13.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:13 smithi043 ceph-mon[41699]: pgmap v1640: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:24:15.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:15 smithi155 ceph-mon[33846]: pgmap v1641: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:15.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:15 smithi043 ceph-mon[30062]: pgmap v1641: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:15.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:15 smithi043 ceph-mon[41699]: pgmap v1641: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:17.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:17 smithi155 ceph-mon[33846]: pgmap v1642: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:17.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:17 smithi043 ceph-mon[30062]: pgmap v1642: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:17.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:17 smithi043 ceph-mon[41699]: pgmap v1642: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:18.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:24:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:24:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:24:19.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:19 smithi155 ceph-mon[33846]: pgmap v1643: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:19.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:24:19 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:24:19.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:19 smithi043 ceph-mon[30062]: pgmap v1643: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:19.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:19 smithi043 ceph-mon[41699]: pgmap v1643: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:20.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:20 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:24:20.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:20 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:24:20.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:20 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:24:20.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:20 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:24:20.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:20 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:24:20.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:20 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:24:21.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:21 smithi155 ceph-mon[33846]: pgmap v1644: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:21.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:21 smithi043 ceph-mon[30062]: pgmap v1644: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:21.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:21 smithi043 ceph-mon[41699]: pgmap v1644: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:23.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:23 smithi155 ceph-mon[33846]: pgmap v1645: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:23.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:23 smithi043 ceph-mon[30062]: pgmap v1645: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:23.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:23 smithi043 ceph-mon[41699]: pgmap v1645: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:25.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:25 smithi155 ceph-mon[33846]: pgmap v1646: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:25.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:25 smithi043 ceph-mon[30062]: pgmap v1646: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:25.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:25 smithi043 ceph-mon[41699]: pgmap v1646: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:27.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:27 smithi155 ceph-mon[33846]: pgmap v1647: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:27.986 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:27 smithi043 ceph-mon[30062]: pgmap v1647: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:27.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:27 smithi043 ceph-mon[41699]: pgmap v1647: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:28.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:24:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:24:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:24:29.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:29 smithi155 ceph-mon[33846]: pgmap v1648: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:29.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:24:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:24:29.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:29 smithi043 ceph-mon[30062]: pgmap v1648: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:29.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:29 smithi043 ceph-mon[41699]: pgmap v1648: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:30.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:30 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:24:30.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:30 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:24:30.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:30 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:24:31.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:31 smithi043 ceph-mon[30062]: pgmap v1649: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:31.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:31 smithi043 ceph-mon[41699]: pgmap v1649: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:31 smithi155 ceph-mon[33846]: pgmap v1649: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:33.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:33 smithi043 ceph-mon[30062]: pgmap v1650: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:33.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:33 smithi043 ceph-mon[41699]: pgmap v1650: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:34.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:33 smithi155 ceph-mon[33846]: pgmap v1650: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:24:34.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:34 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:24:34.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:34 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:24:35.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:34 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:24:35.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:35 smithi043 ceph-mon[30062]: pgmap v1651: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:24:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:35 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:24:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:24:35.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:35 smithi043 ceph-mon[41699]: pgmap v1651: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:35.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:35.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:24:35.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:35 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:24:35.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:24:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:35 smithi155 ceph-mon[33846]: pgmap v1651: 292 pgs: 292 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:24:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:24:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:35 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:24:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:24:37.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:36 smithi155 ceph-mon[33846]: osdmap e752: 8 total, 8 up, 8 in 2024-03-21T13:24:37.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:36 smithi043 ceph-mon[30062]: osdmap e752: 8 total, 8 up, 8 in 2024-03-21T13:24:37.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:36 smithi043 ceph-mon[41699]: osdmap e752: 8 total, 8 up, 8 in 2024-03-21T13:24:37.783 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.783 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.783 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.783 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.783 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.783 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.783 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.783 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.783 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.783 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.783 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.783 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.784 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.784 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.784 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.784 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.784 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.784 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.784 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.784 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.784 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.784 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.784 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.784 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.784 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.785 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.785 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.785 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.785 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.785 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.785 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.785 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.785 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.785 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.785 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.785 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.785 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.785 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.785 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.785 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.786 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.786 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.786 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.786 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.786 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.786 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.786 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.786 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.786 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.786 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.786 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.786 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.786 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 1 sec 2024-03-21T13:24:37.786 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: started 512 aios 2024-03-21T13:24:37.787 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585521d6600 2024-03-21T13:24:37.787 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552516200 2024-03-21T13:24:37.787 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552516bd0 2024-03-21T13:24:37.787 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251b620 2024-03-21T13:24:37.787 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251bc80 2024-03-21T13:24:37.787 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251c2a0 2024-03-21T13:24:37.787 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251c190 2024-03-21T13:24:37.787 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251c7d0 2024-03-21T13:24:37.787 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251cc80 2024-03-21T13:24:37.787 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251cd90 2024-03-21T13:24:37.787 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251d2a0 2024-03-21T13:24:37.787 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251d400 2024-03-21T13:24:37.787 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525166f0 2024-03-21T13:24:37.788 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251cfd0 2024-03-21T13:24:37.788 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251d0e0 2024-03-21T13:24:37.788 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251d510 2024-03-21T13:24:37.788 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251d620 2024-03-21T13:24:37.788 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251dbe0 2024-03-21T13:24:37.788 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251d730 2024-03-21T13:24:37.788 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251dcf0 2024-03-21T13:24:37.788 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251d880 2024-03-21T13:24:37.788 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251da00 2024-03-21T13:24:37.788 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251de00 2024-03-21T13:24:37.788 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251df10 2024-03-21T13:24:37.788 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251e4d0 2024-03-21T13:24:37.788 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251e020 2024-03-21T13:24:37.788 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251c8e0 2024-03-21T13:24:37.788 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251c9f0 2024-03-21T13:24:37.789 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251cb00 2024-03-21T13:24:37.789 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251ea90 2024-03-21T13:24:37.789 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251e5e0 2024-03-21T13:24:37.789 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251e730 2024-03-21T13:24:37.789 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251e840 2024-03-21T13:24:37.789 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251eba0 2024-03-21T13:24:37.789 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251f050 2024-03-21T13:24:37.789 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251ecb0 2024-03-21T13:24:37.789 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251edc0 2024-03-21T13:24:37.789 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251eed0 2024-03-21T13:24:37.789 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251e950 2024-03-21T13:24:37.789 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251f1d0 2024-03-21T13:24:37.789 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251f680 2024-03-21T13:24:37.789 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251f7b0 2024-03-21T13:24:37.789 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251fd70 2024-03-21T13:24:37.790 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251f2e0 2024-03-21T13:24:37.790 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251f410 2024-03-21T13:24:37.790 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251f520 2024-03-21T13:24:37.790 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251fe80 2024-03-21T13:24:37.790 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251f8c0 2024-03-21T13:24:37.790 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251f9f0 2024-03-21T13:24:37.790 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251fb00 2024-03-21T13:24:37.790 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251fc30 2024-03-21T13:24:37.790 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552520480 2024-03-21T13:24:37.790 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525205b0 2024-03-21T13:24:37.790 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525206c0 2024-03-21T13:24:37.790 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525207f0 2024-03-21T13:24:37.790 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552520950 2024-03-21T13:24:37.791 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251ffd0 2024-03-21T13:24:37.791 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552520100 2024-03-21T13:24:37.791 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552520250 2024-03-21T13:24:37.791 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552520a80 2024-03-21T13:24:37.791 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552520f50 2024-03-21T13:24:37.791 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552521080 2024-03-21T13:24:37.791 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552520360 2024-03-21T13:24:37.791 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552520bb0 2024-03-21T13:24:37.791 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552520d00 2024-03-21T13:24:37.791 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552520e30 2024-03-21T13:24:37.791 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552521660 2024-03-21T13:24:37.791 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525211b0 2024-03-21T13:24:37.791 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251e130 2024-03-21T13:24:37.792 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855251e260 2024-03-21T13:24:37.792 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552521790 2024-03-21T13:24:37.792 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552521c80 2024-03-21T13:24:37.792 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552521d90 2024-03-21T13:24:37.792 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525212c0 2024-03-21T13:24:37.792 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552521430 2024-03-21T13:24:37.792 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552522350 2024-03-21T13:24:37.792 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552521ea0 2024-03-21T13:24:37.792 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552521ff0 2024-03-21T13:24:37.792 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552522120 2024-03-21T13:24:37.792 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552522460 2024-03-21T13:24:37.792 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552522910 2024-03-21T13:24:37.792 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552522a60 2024-03-21T13:24:37.792 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552523020 2024-03-21T13:24:37.792 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552522570 2024-03-21T13:24:37.793 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525226a0 2024-03-21T13:24:37.793 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525227d0 2024-03-21T13:24:37.793 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552522230 2024-03-21T13:24:37.793 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552522b70 2024-03-21T13:24:37.793 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525218e0 2024-03-21T13:24:37.793 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552521a30 2024-03-21T13:24:37.793 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552521b40 2024-03-21T13:24:37.793 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525235e0 2024-03-21T13:24:37.793 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552523130 2024-03-21T13:24:37.793 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552523730 2024-03-21T13:24:37.794 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552523840 2024-03-21T13:24:37.794 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552523e00 2024-03-21T13:24:37.794 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552523240 2024-03-21T13:24:37.794 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552523350 2024-03-21T13:24:37.794 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525234a0 2024-03-21T13:24:37.794 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552523f50 2024-03-21T13:24:37.794 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552523950 2024-03-21T13:24:37.794 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552522c80 2024-03-21T13:24:37.794 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552522dd0 2024-03-21T13:24:37.794 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525240a0 2024-03-21T13:24:37.794 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552524590 2024-03-21T13:24:37.794 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525246c0 2024-03-21T13:24:37.794 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552522f00 2024-03-21T13:24:37.794 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525247f0 2024-03-21T13:24:37.794 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552524980 2024-03-21T13:24:37.795 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552524f80 2024-03-21T13:24:37.795 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552523a60 2024-03-21T13:24:37.795 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552523b90 2024-03-21T13:24:37.795 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552523ce0 2024-03-21T13:24:37.795 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525250d0 2024-03-21T13:24:37.795 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552524ad0 2024-03-21T13:24:37.795 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552524c00 2024-03-21T13:24:37.795 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552524d50 2024-03-21T13:24:37.795 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552525240 2024-03-21T13:24:37.795 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552525730 2024-03-21T13:24:37.796 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552525860 2024-03-21T13:24:37.796 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552525e40 2024-03-21T13:24:37.796 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552525390 2024-03-21T13:24:37.796 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525254a0 2024-03-21T13:24:37.796 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525255d0 2024-03-21T13:24:37.796 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552524e60 2024-03-21T13:24:37.796 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552525990 2024-03-21T13:24:37.796 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552525b00 2024-03-21T13:24:37.796 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552525c30 2024-03-21T13:24:37.796 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552525f90 2024-03-21T13:24:37.796 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552526460 2024-03-21T13:24:37.796 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552526590 2024-03-21T13:24:37.796 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552526b70 2024-03-21T13:24:37.797 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525260c0 2024-03-21T13:24:37.797 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552526210 2024-03-21T13:24:37.797 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525266c0 2024-03-21T13:24:37.797 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552526810 2024-03-21T13:24:37.797 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552526920 2024-03-21T13:24:37.797 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552526a50 2024-03-21T13:24:37.797 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552527170 2024-03-21T13:24:37.797 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552526cc0 2024-03-21T13:24:37.797 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552526df0 2024-03-21T13:24:37.797 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552526f20 2024-03-21T13:24:37.797 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525272a0 2024-03-21T13:24:37.797 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552527750 2024-03-21T13:24:37.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552527880 2024-03-21T13:24:37.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525241f0 2024-03-21T13:24:37.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552524320 2024-03-21T13:24:37.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552526340 2024-03-21T13:24:37.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552524450 2024-03-21T13:24:37.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552527e60 2024-03-21T13:24:37.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525279b0 2024-03-21T13:24:37.798 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552527b00 2024-03-21T13:24:37.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552527c70 2024-03-21T13:24:37.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552527f90 2024-03-21T13:24:37.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552528140 2024-03-21T13:24:37.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552528720 2024-03-21T13:24:37.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552528270 2024-03-21T13:24:37.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525273b0 2024-03-21T13:24:37.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525274e0 2024-03-21T13:24:37.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552527610 2024-03-21T13:24:37.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552528d00 2024-03-21T13:24:37.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552528850 2024-03-21T13:24:37.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552528980 2024-03-21T13:24:37.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552528ab0 2024-03-21T13:24:37.799 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552528e30 2024-03-21T13:24:37.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552529300 2024-03-21T13:24:37.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552529430 2024-03-21T13:24:37.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552528380 2024-03-21T13:24:37.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525284b0 2024-03-21T13:24:37.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525285e0 2024-03-21T13:24:37.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552529560 2024-03-21T13:24:37.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552529a50 2024-03-21T13:24:37.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552529b60 2024-03-21T13:24:37.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552528f60 2024-03-21T13:24:37.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552529070 2024-03-21T13:24:37.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552528be0 2024-03-21T13:24:37.800 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525291a0 2024-03-21T13:24:37.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252a120 2024-03-21T13:24:37.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552529c70 2024-03-21T13:24:37.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552529dc0 2024-03-21T13:24:37.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552529f10 2024-03-21T13:24:37.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252a270 2024-03-21T13:24:37.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252a720 2024-03-21T13:24:37.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252a890 2024-03-21T13:24:37.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552529670 2024-03-21T13:24:37.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525297a0 2024-03-21T13:24:37.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525298d0 2024-03-21T13:24:37.801 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252a9e0 2024-03-21T13:24:37.802 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252ae90 2024-03-21T13:24:37.802 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252afe0 2024-03-21T13:24:37.802 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252a380 2024-03-21T13:24:37.802 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252a4d0 2024-03-21T13:24:37.802 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252a5e0 2024-03-21T13:24:37.802 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252b110 2024-03-21T13:24:37.802 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252b5e0 2024-03-21T13:24:37.802 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252b730 2024-03-21T13:24:37.802 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252b220 2024-03-21T13:24:37.802 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252b370 2024-03-21T13:24:37.802 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252b4c0 2024-03-21T13:24:37.802 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252b880 2024-03-21T13:24:37.803 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252bd50 2024-03-21T13:24:37.803 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252be80 2024-03-21T13:24:37.803 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252aaf0 2024-03-21T13:24:37.803 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252ac20 2024-03-21T13:24:37.803 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252ad50 2024-03-21T13:24:37.803 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252bfd0 2024-03-21T13:24:37.803 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252c4e0 2024-03-21T13:24:37.803 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252c630 2024-03-21T13:24:37.803 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252b9b0 2024-03-21T13:24:37.803 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252bae0 2024-03-21T13:24:37.803 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252bbf0 2024-03-21T13:24:37.803 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252c740 2024-03-21T13:24:37.804 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252cbf0 2024-03-21T13:24:37.804 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252cd00 2024-03-21T13:24:37.804 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252d300 2024-03-21T13:24:37.804 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252c850 2024-03-21T13:24:37.804 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252c9a0 2024-03-21T13:24:37.804 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252ce50 2024-03-21T13:24:37.804 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252cf80 2024-03-21T13:24:37.804 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252d0b0 2024-03-21T13:24:37.804 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252d430 2024-03-21T13:24:37.804 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252d8e0 2024-03-21T13:24:37.805 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252da10 2024-03-21T13:24:37.805 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252c120 2024-03-21T13:24:37.805 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252c250 2024-03-21T13:24:37.805 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252c380 2024-03-21T13:24:37.805 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252db40 2024-03-21T13:24:37.805 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252e030 2024-03-21T13:24:37.805 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252e160 2024-03-21T13:24:37.805 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252d540 2024-03-21T13:24:37.805 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252d670 2024-03-21T13:24:37.805 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252d7a0 2024-03-21T13:24:37.805 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252e2b0 2024-03-21T13:24:37.806 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252e760 2024-03-21T13:24:37.806 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252e8d0 2024-03-21T13:24:37.806 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252dc70 2024-03-21T13:24:37.806 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252dd80 2024-03-21T13:24:37.806 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252ded0 2024-03-21T13:24:37.806 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252ea20 2024-03-21T13:24:37.806 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252eef0 2024-03-21T13:24:37.806 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252f020 2024-03-21T13:24:37.806 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252e3c0 2024-03-21T13:24:37.806 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252e510 2024-03-21T13:24:37.806 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252e620 2024-03-21T13:24:37.806 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252f150 2024-03-21T13:24:37.806 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252f600 2024-03-21T13:24:37.806 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252f730 2024-03-21T13:24:37.807 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252fd30 2024-03-21T13:24:37.807 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252f260 2024-03-21T13:24:37.807 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252f390 2024-03-21T13:24:37.807 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252f4c0 2024-03-21T13:24:37.807 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252fe60 2024-03-21T13:24:37.807 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252f880 2024-03-21T13:24:37.807 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252eb50 2024-03-21T13:24:37.807 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252eca0 2024-03-21T13:24:37.807 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252edd0 2024-03-21T13:24:37.807 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552530460 2024-03-21T13:24:37.807 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252ffb0 2024-03-21T13:24:37.807 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525300e0 2024-03-21T13:24:37.807 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552530230 2024-03-21T13:24:37.807 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552530570 2024-03-21T13:24:37.807 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552530a40 2024-03-21T13:24:37.807 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552530b70 2024-03-21T13:24:37.808 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252f990 2024-03-21T13:24:37.808 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252fac0 2024-03-21T13:24:37.808 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855252fbf0 2024-03-21T13:24:37.808 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552530ca0 2024-03-21T13:24:37.808 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552531150 2024-03-21T13:24:37.808 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525312a0 2024-03-21T13:24:37.808 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552530680 2024-03-21T13:24:37.808 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525307d0 2024-03-21T13:24:37.808 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552530900 2024-03-21T13:24:37.808 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525313f0 2024-03-21T13:24:37.808 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525318c0 2024-03-21T13:24:37.808 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552531a10 2024-03-21T13:24:37.808 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552531ff0 2024-03-21T13:24:37.808 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552531520 2024-03-21T13:24:37.808 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552531650 2024-03-21T13:24:37.809 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552531780 2024-03-21T13:24:37.809 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552532140 2024-03-21T13:24:37.809 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552531b40 2024-03-21T13:24:37.809 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552530db0 2024-03-21T13:24:37.809 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552530f00 2024-03-21T13:24:37.809 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552532270 2024-03-21T13:24:37.809 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552532740 2024-03-21T13:24:37.809 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552532870 2024-03-21T13:24:37.809 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552532e30 2024-03-21T13:24:37.809 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525323a0 2024-03-21T13:24:37.809 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552531030 2024-03-21T13:24:37.809 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525324b0 2024-03-21T13:24:37.809 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525325e0 2024-03-21T13:24:37.809 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552532980 2024-03-21T13:24:37.809 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552532ab0 2024-03-21T13:24:37.810 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552532c00 2024-03-21T13:24:37.810 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552532f80 2024-03-21T13:24:37.810 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552533470 2024-03-21T13:24:37.810 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525335a0 2024-03-21T13:24:37.810 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552533b80 2024-03-21T13:24:37.810 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525330d0 2024-03-21T13:24:37.810 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552533200 2024-03-21T13:24:37.810 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552533330 2024-03-21T13:24:37.810 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552533cb0 2024-03-21T13:24:37.810 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525336d0 2024-03-21T13:24:37.810 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552533800 2024-03-21T13:24:37.810 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552533950 2024-03-21T13:24:37.810 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552533de0 2024-03-21T13:24:37.810 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525342b0 2024-03-21T13:24:37.810 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525343e0 2024-03-21T13:24:37.811 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552531c50 2024-03-21T13:24:37.811 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552531d80 2024-03-21T13:24:37.811 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552531eb0 2024-03-21T13:24:37.811 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552534510 2024-03-21T13:24:37.811 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552534a00 2024-03-21T13:24:37.811 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552534b30 2024-03-21T13:24:37.811 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552533ef0 2024-03-21T13:24:37.811 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552534040 2024-03-21T13:24:37.811 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552534170 2024-03-21T13:24:37.811 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552534c60 2024-03-21T13:24:37.811 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552535150 2024-03-21T13:24:37.811 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552535280 2024-03-21T13:24:37.811 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552535860 2024-03-21T13:24:37.811 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552534db0 2024-03-21T13:24:37.811 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552534f00 2024-03-21T13:24:37.811 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552535030 2024-03-21T13:24:37.812 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552535970 2024-03-21T13:24:37.812 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525353b0 2024-03-21T13:24:37.812 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525354e0 2024-03-21T13:24:37.812 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552535650 2024-03-21T13:24:37.812 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552535aa0 2024-03-21T13:24:37.812 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552535fc0 2024-03-21T13:24:37.812 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552536140 2024-03-21T13:24:37.812 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552536720 2024-03-21T13:24:37.812 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552535bd0 2024-03-21T13:24:37.812 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552535ce0 2024-03-21T13:24:37.812 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552535df0 2024-03-21T13:24:37.812 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552536850 2024-03-21T13:24:37.812 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552536270 2024-03-21T13:24:37.812 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525363a0 2024-03-21T13:24:37.812 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525364d0 2024-03-21T13:24:37.812 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552536600 2024-03-21T13:24:37.813 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552536e30 2024-03-21T13:24:37.813 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552536980 2024-03-21T13:24:37.813 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552536ad0 2024-03-21T13:24:37.813 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552536c00 2024-03-21T13:24:37.813 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552536f60 2024-03-21T13:24:37.813 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552537430 2024-03-21T13:24:37.813 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552537560 2024-03-21T13:24:37.813 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552537b40 2024-03-21T13:24:37.813 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552537090 2024-03-21T13:24:37.813 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525371c0 2024-03-21T13:24:37.813 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525372f0 2024-03-21T13:24:37.813 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552537c70 2024-03-21T13:24:37.813 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552537690 2024-03-21T13:24:37.813 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525377c0 2024-03-21T13:24:37.813 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525378f0 2024-03-21T13:24:37.814 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552537a20 2024-03-21T13:24:37.814 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552538250 2024-03-21T13:24:37.814 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552537da0 2024-03-21T13:24:37.814 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552537ef0 2024-03-21T13:24:37.814 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552538020 2024-03-21T13:24:37.814 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552538380 2024-03-21T13:24:37.814 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552538850 2024-03-21T13:24:37.814 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552538980 2024-03-21T13:24:37.814 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552534640 2024-03-21T13:24:37.814 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552534770 2024-03-21T13:24:37.814 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525348c0 2024-03-21T13:24:37.814 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552538ab0 2024-03-21T13:24:37.814 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552538fa0 2024-03-21T13:24:37.814 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525390f0 2024-03-21T13:24:37.814 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552538130 2024-03-21T13:24:37.814 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552538be0 2024-03-21T13:24:37.815 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552538d10 2024-03-21T13:24:37.815 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552538e80 2024-03-21T13:24:37.815 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525396d0 2024-03-21T13:24:37.815 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552539220 2024-03-21T13:24:37.815 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552539370 2024-03-21T13:24:37.815 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525394a0 2024-03-21T13:24:37.815 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552539800 2024-03-21T13:24:37.815 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552539cd0 2024-03-21T13:24:37.815 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552539e00 2024-03-21T13:24:37.815 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525395b0 2024-03-21T13:24:37.815 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552539f30 2024-03-21T13:24:37.815 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253a060 2024-03-21T13:24:37.815 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552539930 2024-03-21T13:24:37.815 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552539a80 2024-03-21T13:24:37.815 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253a640 2024-03-21T13:24:37.815 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253a190 2024-03-21T13:24:37.816 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253a2c0 2024-03-21T13:24:37.816 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253a3f0 2024-03-21T13:24:37.816 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253a520 2024-03-21T13:24:37.816 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253ac20 2024-03-21T13:24:37.816 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253a770 2024-03-21T13:24:37.816 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253a8c0 2024-03-21T13:24:37.816 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253a9f0 2024-03-21T13:24:37.816 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253ad50 2024-03-21T13:24:37.816 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253b220 2024-03-21T13:24:37.816 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253b350 2024-03-21T13:24:37.816 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525384b0 2024-03-21T13:24:37.816 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525385e0 2024-03-21T13:24:37.816 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552538710 2024-03-21T13:24:37.816 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253b480 2024-03-21T13:24:37.816 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253b950 2024-03-21T13:24:37.816 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253baa0 2024-03-21T13:24:37.817 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253ae80 2024-03-21T13:24:37.817 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253afb0 2024-03-21T13:24:37.817 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253b0e0 2024-03-21T13:24:37.817 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253bbd0 2024-03-21T13:24:37.817 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253c0a0 2024-03-21T13:24:37.817 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253c1d0 2024-03-21T13:24:37.817 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552539bb0 2024-03-21T13:24:37.817 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253c320 2024-03-21T13:24:37.817 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253c470 2024-03-21T13:24:37.817 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253bd00 2024-03-21T13:24:37.817 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253be30 2024-03-21T13:24:37.817 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253bf60 2024-03-21T13:24:37.817 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253c5c0 2024-03-21T13:24:37.817 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253ca90 2024-03-21T13:24:37.817 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253cbe0 2024-03-21T13:24:37.817 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253b5b0 2024-03-21T13:24:37.818 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253b6e0 2024-03-21T13:24:37.818 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253b810 2024-03-21T13:24:37.818 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253cd10 2024-03-21T13:24:37.818 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253d1e0 2024-03-21T13:24:37.818 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253d310 2024-03-21T13:24:37.818 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253c6f0 2024-03-21T13:24:37.818 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253c820 2024-03-21T13:24:37.818 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253c950 2024-03-21T13:24:37.818 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253d460 2024-03-21T13:24:37.818 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253d930 2024-03-21T13:24:37.818 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253da80 2024-03-21T13:24:37.818 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253ce20 2024-03-21T13:24:37.818 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253cf70 2024-03-21T13:24:37.818 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253d0a0 2024-03-21T13:24:37.818 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253dbb0 2024-03-21T13:24:37.818 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253e080 2024-03-21T13:24:37.819 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253e1b0 2024-03-21T13:24:37.819 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253d590 2024-03-21T13:24:37.819 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253d6c0 2024-03-21T13:24:37.819 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253d7f0 2024-03-21T13:24:37.819 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253e300 2024-03-21T13:24:37.819 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253e820 2024-03-21T13:24:37.819 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253e970 2024-03-21T13:24:37.819 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253dce0 2024-03-21T13:24:37.819 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253de10 2024-03-21T13:24:37.819 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253df40 2024-03-21T13:24:37.819 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253eaa0 2024-03-21T13:24:37.820 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253ef70 2024-03-21T13:24:38.134 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:24:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:24:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:24:38.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:37 smithi043 ceph-mon[30062]: pgmap v1653: 260 pgs: 260 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:24:38.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:37 smithi043 ceph-mon[30062]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:24:38.135 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:37 smithi043 ceph-mon[30062]: osdmap e753: 8 total, 8 up, 8 in 2024-03-21T13:24:38.135 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:37 smithi043 ceph-mon[41699]: pgmap v1653: 260 pgs: 260 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:24:38.135 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:37 smithi043 ceph-mon[41699]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:24:38.135 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:37 smithi043 ceph-mon[41699]: osdmap e753: 8 total, 8 up, 8 in 2024-03-21T13:24:38.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:37 smithi155 ceph-mon[33846]: pgmap v1653: 260 pgs: 260 active+clean; 582 KiB data, 832 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:24:38.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:37 smithi155 ceph-mon[33846]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:24:38.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:37 smithi155 ceph-mon[33846]: osdmap e753: 8 total, 8 up, 8 in 2024-03-21T13:24:39.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:38 smithi155 ceph-mon[33846]: osdmap e754: 8 total, 8 up, 8 in 2024-03-21T13:24:39.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:38 smithi043 ceph-mon[30062]: osdmap e754: 8 total, 8 up, 8 in 2024-03-21T13:24:39.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:38 smithi043 ceph-mon[41699]: osdmap e754: 8 total, 8 up, 8 in 2024-03-21T13:24:40.108 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:39 smithi155 ceph-mon[33846]: pgmap v1656: 260 pgs: 260 active+clean; 582 KiB data, 836 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:24:40.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:39 smithi155 ceph-mon[33846]: osdmap e755: 8 total, 8 up, 8 in 2024-03-21T13:24:40.109 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:39 smithi155 ceph-mon[33846]: osdmap e756: 8 total, 8 up, 8 in 2024-03-21T13:24:40.109 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:24:39 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:24:40.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:39 smithi043 ceph-mon[30062]: pgmap v1656: 260 pgs: 260 active+clean; 582 KiB data, 836 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:24:40.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:39 smithi043 ceph-mon[30062]: osdmap e755: 8 total, 8 up, 8 in 2024-03-21T13:24:40.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:39 smithi043 ceph-mon[30062]: osdmap e756: 8 total, 8 up, 8 in 2024-03-21T13:24:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:39 smithi043 ceph-mon[41699]: pgmap v1656: 260 pgs: 260 active+clean; 582 KiB data, 836 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:24:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:39 smithi043 ceph-mon[41699]: osdmap e755: 8 total, 8 up, 8 in 2024-03-21T13:24:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:39 smithi043 ceph-mon[41699]: osdmap e756: 8 total, 8 up, 8 in 2024-03-21T13:24:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:40 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:24:41.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:40 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:24:41.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:40 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:24:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:41 smithi155 ceph-mon[33846]: pgmap v1659: 228 pgs: 32 unknown, 196 active+clean; 582 KiB data, 836 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:41 smithi155 ceph-mon[33846]: osdmap e757: 8 total, 8 up, 8 in 2024-03-21T13:24:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:41 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:24:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:41 smithi155 ceph-mon[33846]: from='client.26368 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:24:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:41 smithi043 ceph-mon[30062]: pgmap v1659: 228 pgs: 32 unknown, 196 active+clean; 582 KiB data, 836 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:41 smithi043 ceph-mon[30062]: osdmap e757: 8 total, 8 up, 8 in 2024-03-21T13:24:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:41 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:24:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:41 smithi043 ceph-mon[30062]: from='client.26368 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:24:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:41 smithi043 ceph-mon[41699]: pgmap v1659: 228 pgs: 32 unknown, 196 active+clean; 582 KiB data, 836 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:41 smithi043 ceph-mon[41699]: osdmap e757: 8 total, 8 up, 8 in 2024-03-21T13:24:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:41 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:24:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:41 smithi043 ceph-mon[41699]: from='client.26368 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:24:42.824 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525 watch_notify: Running main() from gmock_main.cc 2024-03-21T13:24:42.824 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [==========] Running 2 tests from 1 test suite. 2024-03-21T13:24:42.824 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [----------] Global test environment set-up. 2024-03-21T13:24:42.824 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify 2024-03-21T13:24:42.824 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotify 2024-03-21T13:24:42.824 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: handle_notify cookie 93936246760368 notify_id 3225520439302 notifier_gid 24916 2024-03-21T13:24:42.824 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotify (1809727 ms) 2024-03-21T13:24:42.824 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotifyTimeout 2024-03-21T13:24:42.824 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: Trying... 2024-03-21T13:24:42.824 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: handle_notify cookie 93936259454816 notify_id 3238405341187 notifier_gid 15579 2024-03-21T13:24:42.824 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: Waiting for 3.000000000s 2024-03-21T13:24:42.825 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: Timed out. 2024-03-21T13:24:42.825 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: Flushing... 2024-03-21T13:24:42.825 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: Flushed... 2024-03-21T13:24:42.825 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotifyTimeout (7036 ms) 2024-03-21T13:24:42.825 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify (1816763 ms total) 2024-03-21T13:24:42.825 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: 2024-03-21T13:24:42.825 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [----------] Global test environment tear-down 2024-03-21T13:24:42.825 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [==========] 2 tests from 1 test suite ran. (1816763 ms total) 2024-03-21T13:24:42.825 INFO:tasks.workunit.client.0.smithi043.stdout: watch_notify: [ PASSED ] 2 tests. 2024-03-21T13:24:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:42 smithi155 ceph-mon[33846]: from='client.26368 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:24:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:42 smithi155 ceph-mon[33846]: osdmap e758: 8 total, 8 up, 8 in 2024-03-21T13:24:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-03-21T13:24:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:42 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:24:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:42 smithi155 ceph-mon[33846]: from='client.26368 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:24:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:42 smithi155 ceph-mon[33846]: from='client.26368 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-21T13:24:43.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:42 smithi155 ceph-mon[33846]: osdmap e759: 8 total, 8 up, 8 in 2024-03-21T13:24:43.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:42 smithi043 ceph-mon[30062]: from='client.26368 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:24:43.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:42 smithi043 ceph-mon[30062]: osdmap e758: 8 total, 8 up, 8 in 2024-03-21T13:24:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-03-21T13:24:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:42 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:24:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:42 smithi043 ceph-mon[30062]: from='client.26368 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:24:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:42 smithi043 ceph-mon[30062]: from='client.26368 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-21T13:24:43.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:42 smithi043 ceph-mon[30062]: osdmap e759: 8 total, 8 up, 8 in 2024-03-21T13:24:43.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:42 smithi043 ceph-mon[41699]: from='client.26368 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-21T13:24:43.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:42 smithi043 ceph-mon[41699]: osdmap e758: 8 total, 8 up, 8 in 2024-03-21T13:24:43.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-03-21T13:24:43.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:42 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:24:43.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:42 smithi043 ceph-mon[41699]: from='client.26368 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:24:43.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:42 smithi043 ceph-mon[41699]: from='client.26368 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-21T13:24:43.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:42 smithi043 ceph-mon[41699]: osdmap e759: 8 total, 8 up, 8 in 2024-03-21T13:24:44.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:43 smithi155 ceph-mon[33846]: pgmap v1662: 260 pgs: 64 unknown, 196 active+clean; 582 KiB data, 836 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:24:44.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "app2"}]: dispatch 2024-03-21T13:24:44.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:24:44.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:43 smithi155 ceph-mon[33846]: from='client.26368 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:24:44.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:43 smithi155 ceph-mon[33846]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:24:44.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:43 smithi043 ceph-mon[30062]: pgmap v1662: 260 pgs: 64 unknown, 196 active+clean; 582 KiB data, 836 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:24:44.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "app2"}]: dispatch 2024-03-21T13:24:44.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:24:44.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:43 smithi043 ceph-mon[30062]: from='client.26368 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:24:44.235 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:43 smithi043 ceph-mon[30062]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:24:44.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:43 smithi043 ceph-mon[41699]: pgmap v1662: 260 pgs: 64 unknown, 196 active+clean; 582 KiB data, 836 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:24:44.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "app2"}]: dispatch 2024-03-21T13:24:44.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:24:44.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:43 smithi043 ceph-mon[41699]: from='client.26368 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-21T13:24:44.236 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:43 smithi043 ceph-mon[41699]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:24:45.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:44 smithi155 ceph-mon[33846]: from='client.26368 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-21T13:24:45.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:44 smithi155 ceph-mon[33846]: osdmap e760: 8 total, 8 up, 8 in 2024-03-21T13:24:45.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:44 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81877-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-03-21T13:24:45.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:44 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-21T13:24:45.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:44 smithi155 ceph-mon[33846]: from='client.26368 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-21T13:24:45.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:44 smithi043 ceph-mon[30062]: from='client.26368 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-21T13:24:45.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:44 smithi043 ceph-mon[30062]: osdmap e760: 8 total, 8 up, 8 in 2024-03-21T13:24:45.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:44 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81877-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-03-21T13:24:45.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:44 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-21T13:24:45.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:44 smithi043 ceph-mon[30062]: from='client.26368 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-21T13:24:45.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:44 smithi043 ceph-mon[41699]: from='client.26368 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi043-81877-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-21T13:24:45.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:44 smithi043 ceph-mon[41699]: osdmap e760: 8 total, 8 up, 8 in 2024-03-21T13:24:45.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:44 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81877-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-03-21T13:24:45.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:44 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-21T13:24:45.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:44 smithi043 ceph-mon[41699]: from='client.26368 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-21T13:24:46.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:45 smithi155 ceph-mon[33846]: pgmap v1665: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:24:46.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:45 smithi155 ceph-mon[33846]: from='client.26368 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-21T13:24:46.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:45 smithi155 ceph-mon[33846]: osdmap e761: 8 total, 8 up, 8 in 2024-03-21T13:24:46.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:45 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-21T13:24:46.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:45 smithi155 ceph-mon[33846]: from='client.26368 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-21T13:24:46.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:45 smithi043 ceph-mon[30062]: pgmap v1665: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:24:46.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:45 smithi043 ceph-mon[30062]: from='client.26368 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-21T13:24:46.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:45 smithi043 ceph-mon[30062]: osdmap e761: 8 total, 8 up, 8 in 2024-03-21T13:24:46.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-21T13:24:46.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:45 smithi043 ceph-mon[30062]: from='client.26368 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-21T13:24:46.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:45 smithi043 ceph-mon[41699]: pgmap v1665: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:24:46.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:45 smithi043 ceph-mon[41699]: from='client.26368 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-21T13:24:46.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:45 smithi043 ceph-mon[41699]: osdmap e761: 8 total, 8 up, 8 in 2024-03-21T13:24:46.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:45 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-21T13:24:46.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:45 smithi043 ceph-mon[41699]: from='client.26368 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-21T13:24:46.880 INFO:tasks.workunit.client.0.smithi043.stdout:3f0c0 2024-03-21T13:24:46.880 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253e430 2024-03-21T13:24:46.880 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253e560 2024-03-21T13:24:46.880 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253e690 2024-03-21T13:24:46.880 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253f1f0 2024-03-21T13:24:46.880 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253f6c0 2024-03-21T13:24:46.880 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253f7f0 2024-03-21T13:24:46.880 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253ebd0 2024-03-21T13:24:46.880 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253ed00 2024-03-21T13:24:46.881 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253ee30 2024-03-21T13:24:46.881 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253f920 2024-03-21T13:24:46.881 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253fe40 2024-03-21T13:24:46.881 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253ff90 2024-03-21T13:24:46.881 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253f320 2024-03-21T13:24:46.881 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253f450 2024-03-21T13:24:46.881 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253f560 2024-03-21T13:24:46.881 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525400c0 2024-03-21T13:24:46.881 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552540590 2024-03-21T13:24:46.881 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525406c0 2024-03-21T13:24:46.881 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253fa30 2024-03-21T13:24:46.881 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253fb80 2024-03-21T13:24:46.881 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x55855253fcd0 2024-03-21T13:24:46.881 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552540810 2024-03-21T13:24:46.881 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552540ce0 2024-03-21T13:24:46.882 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552540e10 2024-03-21T13:24:46.882 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552541410 2024-03-21T13:24:46.882 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552540940 2024-03-21T13:24:46.882 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552540a70 2024-03-21T13:24:46.882 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552540ba0 2024-03-21T13:24:46.882 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552541520 2024-03-21T13:24:46.882 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552540f60 2024-03-21T13:24:46.882 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552541090 2024-03-21T13:24:46.882 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525411e0 2024-03-21T13:24:46.882 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552541670 2024-03-21T13:24:46.882 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552541b40 2024-03-21T13:24:46.882 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552541c70 2024-03-21T13:24:46.882 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525401f0 2024-03-21T13:24:46.882 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552540320 2024-03-21T13:24:46.882 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552540450 2024-03-21T13:24:46.883 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552541da0 2024-03-21T13:24:46.883 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552542270 2024-03-21T13:24:46.883 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525423a0 2024-03-21T13:24:46.883 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525417a0 2024-03-21T13:24:46.883 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525418d0 2024-03-21T13:24:46.883 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552541a00 2024-03-21T13:24:46.883 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525424f0 2024-03-21T13:24:46.883 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525429c0 2024-03-21T13:24:46.883 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552542b10 2024-03-21T13:24:46.883 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552541eb0 2024-03-21T13:24:46.883 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552542000 2024-03-21T13:24:46.883 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552542130 2024-03-21T13:24:46.883 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552542c40 2024-03-21T13:24:46.883 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552543110 2024-03-21T13:24:46.883 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552543240 2024-03-21T13:24:46.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552542600 2024-03-21T13:24:46.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552542750 2024-03-21T13:24:46.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552542880 2024-03-21T13:24:46.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552543390 2024-03-21T13:24:46.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552543860 2024-03-21T13:24:46.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552543990 2024-03-21T13:24:46.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552543f90 2024-03-21T13:24:46.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525434c0 2024-03-21T13:24:46.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525435f0 2024-03-21T13:24:46.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x558552543720 2024-03-21T13:24:46.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: waiting 0x5585525440c0 2024-03-21T13:24:46.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: done. 2024-03-21T13:24:46.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (1812011 ms) 2024-03-21T13:24:46.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (1812011 ms total) 2024-03-21T13:24:46.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: 2024-03-21T13:24:46.884 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-03-21T13:24:46.885 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-03-21T13:24:46.885 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (1 ms) 2024-03-21T13:24:46.885 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-03-21T13:24:46.885 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.Exec (26 ms) 2024-03-21T13:24:46.885 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-03-21T13:24:46.885 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (4 ms) 2024-03-21T13:24:46.885 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-03-21T13:24:46.885 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (4 ms) 2024-03-21T13:24:46.885 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-03-21T13:24:47.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:46 smithi155 ceph-mon[33846]: from='client.26368 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-21T13:24:47.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:46 smithi155 ceph-mon[33846]: osdmap e762: 8 total, 8 up, 8 in 2024-03-21T13:24:47.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:46 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key1"}]: dispatch 2024-03-21T13:24:47.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:46 smithi155 ceph-mon[33846]: from='client.26368 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key1"}]: dispatch 2024-03-21T13:24:47.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:46 smithi155 ceph-mon[33846]: from='client.26368 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key1"}]': finished 2024-03-21T13:24:47.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:46 smithi043 ceph-mon[41699]: from='client.26368 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-21T13:24:47.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:46 smithi043 ceph-mon[41699]: osdmap e762: 8 total, 8 up, 8 in 2024-03-21T13:24:47.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:46 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key1"}]: dispatch 2024-03-21T13:24:47.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:46 smithi043 ceph-mon[41699]: from='client.26368 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key1"}]: dispatch 2024-03-21T13:24:47.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:46 smithi043 ceph-mon[41699]: from='client.26368 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key1"}]': finished 2024-03-21T13:24:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:46 smithi043 ceph-mon[30062]: from='client.26368 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-21T13:24:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:46 smithi043 ceph-mon[30062]: osdmap e762: 8 total, 8 up, 8 in 2024-03-21T13:24:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:46 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2437733278' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key1"}]: dispatch 2024-03-21T13:24:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:46 smithi043 ceph-mon[30062]: from='client.26368 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key1"}]: dispatch 2024-03-21T13:24:47.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:46 smithi043 ceph-mon[30062]: from='client.26368 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi043-81877-1","app":"app1","key":"key1"}]': finished 2024-03-21T13:24:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:47 smithi155 ceph-mon[33846]: pgmap v1668: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:24:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:47 smithi155 ceph-mon[33846]: osdmap e763: 8 total, 8 up, 8 in 2024-03-21T13:24:48.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:47 smithi043 ceph-mon[30062]: pgmap v1668: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:24:48.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:47 smithi043 ceph-mon[30062]: osdmap e763: 8 total, 8 up, 8 in 2024-03-21T13:24:48.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:24:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:24:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:24:48.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:47 smithi043 ceph-mon[41699]: pgmap v1668: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 818 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:24:48.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:47 smithi043 ceph-mon[41699]: osdmap e763: 8 total, 8 up, 8 in 2024-03-21T13:24:49.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:48 smithi155 ceph-mon[33846]: osdmap e764: 8 total, 8 up, 8 in 2024-03-21T13:24:49.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:48 smithi155 ceph-mon[33846]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:24:49.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:48 smithi043 ceph-mon[30062]: osdmap e764: 8 total, 8 up, 8 in 2024-03-21T13:24:49.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:48 smithi043 ceph-mon[30062]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:24:49.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:48 smithi043 ceph-mon[41699]: osdmap e764: 8 total, 8 up, 8 in 2024-03-21T13:24:49.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:48 smithi043 ceph-mon[41699]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:24:50.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:49 smithi155 ceph-mon[33846]: pgmap v1671: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 811 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:50.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:49 smithi155 ceph-mon[33846]: osdmap e765: 8 total, 8 up, 8 in 2024-03-21T13:24:50.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:24:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:24:50.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:49 smithi043 ceph-mon[30062]: pgmap v1671: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 811 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:50.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:49 smithi043 ceph-mon[30062]: osdmap e765: 8 total, 8 up, 8 in 2024-03-21T13:24:50.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:49 smithi043 ceph-mon[41699]: pgmap v1671: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 811 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:50.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:49 smithi043 ceph-mon[41699]: osdmap e765: 8 total, 8 up, 8 in 2024-03-21T13:24:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:50 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:24:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:50 smithi155 ceph-mon[33846]: osdmap e766: 8 total, 8 up, 8 in 2024-03-21T13:24:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:50 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:24:51.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:50 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:24:51.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:50 smithi043 ceph-mon[30062]: osdmap e766: 8 total, 8 up, 8 in 2024-03-21T13:24:51.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:50 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:24:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:50 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:24:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:50 smithi043 ceph-mon[41699]: osdmap e766: 8 total, 8 up, 8 in 2024-03-21T13:24:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:50 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:24:52.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:51 smithi043 ceph-mon[30062]: pgmap v1674: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 811 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:52.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:51 smithi043 ceph-mon[30062]: osdmap e767: 8 total, 8 up, 8 in 2024-03-21T13:24:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:51 smithi043 ceph-mon[41699]: pgmap v1674: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 811 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:51 smithi043 ceph-mon[41699]: osdmap e767: 8 total, 8 up, 8 in 2024-03-21T13:24:52.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:51 smithi155 ceph-mon[33846]: pgmap v1674: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 811 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:52.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:51 smithi155 ceph-mon[33846]: osdmap e767: 8 total, 8 up, 8 in 2024-03-21T13:24:53.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:52 smithi043 ceph-mon[30062]: osdmap e768: 8 total, 8 up, 8 in 2024-03-21T13:24:53.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:52 smithi043 ceph-mon[41699]: osdmap e768: 8 total, 8 up, 8 in 2024-03-21T13:24:53.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:52 smithi155 ceph-mon[33846]: osdmap e768: 8 total, 8 up, 8 in 2024-03-21T13:24:54.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:53 smithi155 ceph-mon[33846]: pgmap v1677: 196 pgs: 196 active+clean; 583 KiB data, 811 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:24:54.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:53 smithi155 ceph-mon[33846]: osdmap e769: 8 total, 8 up, 8 in 2024-03-21T13:24:54.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:53 smithi155 ceph-mon[33846]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:24:54.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:53 smithi043 ceph-mon[30062]: pgmap v1677: 196 pgs: 196 active+clean; 583 KiB data, 811 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:24:54.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:53 smithi043 ceph-mon[30062]: osdmap e769: 8 total, 8 up, 8 in 2024-03-21T13:24:54.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:53 smithi043 ceph-mon[30062]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:24:54.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:53 smithi043 ceph-mon[41699]: pgmap v1677: 196 pgs: 196 active+clean; 583 KiB data, 811 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:24:54.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:53 smithi043 ceph-mon[41699]: osdmap e769: 8 total, 8 up, 8 in 2024-03-21T13:24:54.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:53 smithi043 ceph-mon[41699]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:24:55.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:54 smithi155 ceph-mon[33846]: osdmap e770: 8 total, 8 up, 8 in 2024-03-21T13:24:55.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:54 smithi043 ceph-mon[30062]: osdmap e770: 8 total, 8 up, 8 in 2024-03-21T13:24:55.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:54 smithi043 ceph-mon[41699]: osdmap e770: 8 total, 8 up, 8 in 2024-03-21T13:24:56.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:56 smithi155 ceph-mon[33846]: pgmap v1680: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 854 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:56.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:56 smithi155 ceph-mon[33846]: osdmap e771: 8 total, 8 up, 8 in 2024-03-21T13:24:56.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:56 smithi043 ceph-mon[30062]: pgmap v1680: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 854 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:56.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:56 smithi043 ceph-mon[30062]: osdmap e771: 8 total, 8 up, 8 in 2024-03-21T13:24:56.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:56 smithi043 ceph-mon[41699]: pgmap v1680: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 854 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:56.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:56 smithi043 ceph-mon[41699]: osdmap e771: 8 total, 8 up, 8 in 2024-03-21T13:24:57.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:57 smithi155 ceph-mon[33846]: osdmap e772: 8 total, 8 up, 8 in 2024-03-21T13:24:57.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:57 smithi155 ceph-mon[33846]: pgmap v1683: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 854 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:57.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:57 smithi043 ceph-mon[30062]: osdmap e772: 8 total, 8 up, 8 in 2024-03-21T13:24:57.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:57 smithi043 ceph-mon[30062]: pgmap v1683: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 854 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:57.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:57 smithi043 ceph-mon[41699]: osdmap e772: 8 total, 8 up, 8 in 2024-03-21T13:24:57.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:57 smithi043 ceph-mon[41699]: pgmap v1683: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 854 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:24:58.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:58 smithi155 ceph-mon[33846]: osdmap e773: 8 total, 8 up, 8 in 2024-03-21T13:24:58.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:58 smithi043 ceph-mon[30062]: osdmap e773: 8 total, 8 up, 8 in 2024-03-21T13:24:58.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:24:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:24:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:24:58.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:58 smithi043 ceph-mon[41699]: osdmap e773: 8 total, 8 up, 8 in 2024-03-21T13:24:59.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:59 smithi155 ceph-mon[33846]: osdmap e774: 8 total, 8 up, 8 in 2024-03-21T13:24:59.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:59 smithi155 ceph-mon[33846]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:24:59.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:24:59 smithi155 ceph-mon[33846]: pgmap v1686: 228 pgs: 228 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:24:59.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:59 smithi043 ceph-mon[41699]: osdmap e774: 8 total, 8 up, 8 in 2024-03-21T13:24:59.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:59 smithi043 ceph-mon[41699]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:24:59.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:24:59 smithi043 ceph-mon[41699]: pgmap v1686: 228 pgs: 228 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:24:59.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:59 smithi043 ceph-mon[30062]: osdmap e774: 8 total, 8 up, 8 in 2024-03-21T13:24:59.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:59 smithi043 ceph-mon[30062]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:24:59.485 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:24:59 smithi043 ceph-mon[30062]: pgmap v1686: 228 pgs: 228 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-21T13:25:00.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:24:59 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:25:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:00 smithi155 ceph-mon[33846]: osdmap e775: 8 total, 8 up, 8 in 2024-03-21T13:25:00.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:00 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:25:00.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:00 smithi043 ceph-mon[30062]: osdmap e775: 8 total, 8 up, 8 in 2024-03-21T13:25:00.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:00 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:25:00.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:00 smithi043 ceph-mon[41699]: osdmap e775: 8 total, 8 up, 8 in 2024-03-21T13:25:00.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:00 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:25:01.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:01 smithi155 ceph-mon[33846]: osdmap e776: 8 total, 8 up, 8 in 2024-03-21T13:25:01.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:01 smithi155 ceph-mon[33846]: pgmap v1689: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:25:01.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:01 smithi043 ceph-mon[30062]: osdmap e776: 8 total, 8 up, 8 in 2024-03-21T13:25:01.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:01 smithi043 ceph-mon[30062]: pgmap v1689: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:25:01.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:01 smithi043 ceph-mon[41699]: osdmap e776: 8 total, 8 up, 8 in 2024-03-21T13:25:01.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:01 smithi043 ceph-mon[41699]: pgmap v1689: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:25:02.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:02 smithi155 ceph-mon[33846]: osdmap e777: 8 total, 8 up, 8 in 2024-03-21T13:25:02.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:02 smithi043 ceph-mon[30062]: osdmap e777: 8 total, 8 up, 8 in 2024-03-21T13:25:02.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:02 smithi043 ceph-mon[41699]: osdmap e777: 8 total, 8 up, 8 in 2024-03-21T13:25:03.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:03 smithi155 ceph-mon[33846]: osdmap e778: 8 total, 8 up, 8 in 2024-03-21T13:25:03.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:03 smithi155 ceph-mon[33846]: pgmap v1692: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:25:03.483 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:03 smithi043 ceph-mon[41699]: osdmap e778: 8 total, 8 up, 8 in 2024-03-21T13:25:03.483 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:03 smithi043 ceph-mon[41699]: pgmap v1692: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:25:03.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:03 smithi043 ceph-mon[30062]: osdmap e778: 8 total, 8 up, 8 in 2024-03-21T13:25:03.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:03 smithi043 ceph-mon[30062]: pgmap v1692: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 851 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:25:04.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:04 smithi155 ceph-mon[33846]: osdmap e779: 8 total, 8 up, 8 in 2024-03-21T13:25:04.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:04 smithi043 ceph-mon[30062]: osdmap e779: 8 total, 8 up, 8 in 2024-03-21T13:25:04.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:04 smithi043 ceph-mon[41699]: osdmap e779: 8 total, 8 up, 8 in 2024-03-21T13:25:05.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:05 smithi155 ceph-mon[33846]: osdmap e780: 8 total, 8 up, 8 in 2024-03-21T13:25:05.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:05 smithi155 ceph-mon[33846]: pgmap v1695: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:25:05.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:05 smithi043 ceph-mon[30062]: osdmap e780: 8 total, 8 up, 8 in 2024-03-21T13:25:05.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:05 smithi043 ceph-mon[30062]: pgmap v1695: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:25:05.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:05 smithi043 ceph-mon[41699]: osdmap e780: 8 total, 8 up, 8 in 2024-03-21T13:25:05.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:05 smithi043 ceph-mon[41699]: pgmap v1695: 196 pgs: 196 active+clean; 583 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:25:06.141 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.Applications (5018 ms) 2024-03-21T13:25:06.142 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-03-21T13:25:06.142 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-03-21T13:25:06.142 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-03-21T13:25:06.142 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-03-21T13:25:06.142 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-03-21T13:25:06.142 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (18523 ms) 2024-03-21T13:25:06.142 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] 8 tests from LibRadosMisc (23576 ms total) 2024-03-21T13:25:06.142 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: 2024-03-21T13:25:06.142 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [----------] Global test environment tear-down 2024-03-21T13:25:06.142 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (1840416 ms total) 2024-03-21T13:25:06.142 INFO:tasks.workunit.client.0.smithi043.stdout: api_misc: [ PASSED ] 12 tests. 2024-03-21T13:25:06.145 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:06.145 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82028 2024-03-21T13:25:06.145 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82028 2024-03-21T13:25:06.145 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:06.145 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82515 2024-03-21T13:25:06.145 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82515 2024-03-21T13:25:06.145 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:06.145 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82586 2024-03-21T13:25:06.146 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82586 2024-03-21T13:25:06.146 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:06.146 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81714 2024-03-21T13:25:06.146 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81714 2024-03-21T13:25:06.146 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:06.146 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82338 2024-03-21T13:25:06.146 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82338 2024-03-21T13:25:06.146 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:06.146 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82227 2024-03-21T13:25:06.146 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82227 2024-03-21T13:25:06.146 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:06.146 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81591 2024-03-21T13:25:06.146 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81591 2024-03-21T13:25:06.147 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:06.147 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81655 2024-03-21T13:25:06.147 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81655 2024-03-21T13:25:06.147 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:06.147 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82268 2024-03-21T13:25:06.147 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82268 2024-03-21T13:25:06.147 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:06.147 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81582 2024-03-21T13:25:06.147 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81582 2024-03-21T13:25:06.147 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:06.147 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81603 2024-03-21T13:25:06.147 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81603 2024-03-21T13:25:06.147 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:06.148 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82783 2024-03-21T13:25:06.148 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82783 2024-03-21T13:25:06.148 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:06.148 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81899 2024-03-21T13:25:06.148 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81899 2024-03-21T13:25:06.148 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:06.148 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82892 2024-03-21T13:25:06.148 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82892 2024-03-21T13:25:06.148 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:06.148 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82000 2024-03-21T13:25:06.148 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82000 2024-03-21T13:25:06.148 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:06.148 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82452 2024-03-21T13:25:06.148 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82452 2024-03-21T13:25:06.149 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:06.149 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82936 2024-03-21T13:25:06.149 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82936 2024-03-21T13:25:06.149 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:06.149 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81914 2024-03-21T13:25:06.149 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81914 2024-03-21T13:25:06.149 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:06.149 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82301 2024-03-21T13:25:06.149 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82301 2024-03-21T13:25:06.149 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:06.149 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82756 2024-03-21T13:25:06.149 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82756 2024-03-21T13:25:06.149 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:06.149 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82372 2024-03-21T13:25:06.149 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82372 2024-03-21T13:25:06.150 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:06.150 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82815 2024-03-21T13:25:06.150 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82815 2024-03-21T13:25:06.150 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:06.150 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82204 2024-03-21T13:25:06.150 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82204 2024-03-21T13:25:06.150 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:06.150 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81828 2024-03-21T13:25:06.150 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81828 2024-03-21T13:25:06.150 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:06.150 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82911 2024-03-21T13:25:06.150 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82911 2024-03-21T13:25:06.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:06 smithi155 ceph-mon[33846]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:25:06.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:06 smithi155 ceph-mon[33846]: osdmap e781: 8 total, 8 up, 8 in 2024-03-21T13:25:06.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:06 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:25:06.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:06 smithi043 ceph-mon[30062]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:25:06.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:06 smithi043 ceph-mon[30062]: osdmap e781: 8 total, 8 up, 8 in 2024-03-21T13:25:06.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:06 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:25:06.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:06 smithi043 ceph-mon[41699]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:25:06.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:06 smithi043 ceph-mon[41699]: osdmap e781: 8 total, 8 up, 8 in 2024-03-21T13:25:06.485 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:06 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:25:07.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:07 smithi155 ceph-mon[33846]: osdmap e782: 8 total, 8 up, 8 in 2024-03-21T13:25:07.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:07 smithi155 ceph-mon[33846]: pgmap v1698: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:25:07.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:07 smithi043 ceph-mon[30062]: osdmap e782: 8 total, 8 up, 8 in 2024-03-21T13:25:07.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:07 smithi043 ceph-mon[30062]: pgmap v1698: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:25:07.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:07 smithi043 ceph-mon[41699]: osdmap e782: 8 total, 8 up, 8 in 2024-03-21T13:25:07.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:07 smithi043 ceph-mon[41699]: pgmap v1698: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 835 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-21T13:25:08.385 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:08 smithi043 ceph-mon[30062]: osdmap e783: 8 total, 8 up, 8 in 2024-03-21T13:25:08.385 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:08 smithi043 ceph-mon[41699]: osdmap e783: 8 total, 8 up, 8 in 2024-03-21T13:25:08.385 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:25:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:25:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:25:08.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:08 smithi155 ceph-mon[33846]: osdmap e783: 8 total, 8 up, 8 in 2024-03-21T13:25:09.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:09 smithi155 ceph-mon[33846]: osdmap e784: 8 total, 8 up, 8 in 2024-03-21T13:25:09.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:09 smithi155 ceph-mon[33846]: pgmap v1701: 196 pgs: 196 active+clean; 583 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-03-21T13:25:09.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:09 smithi043 ceph-mon[30062]: osdmap e784: 8 total, 8 up, 8 in 2024-03-21T13:25:09.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:09 smithi043 ceph-mon[30062]: pgmap v1701: 196 pgs: 196 active+clean; 583 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-03-21T13:25:09.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:09 smithi043 ceph-mon[41699]: osdmap e784: 8 total, 8 up, 8 in 2024-03-21T13:25:09.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:09 smithi043 ceph-mon[41699]: pgmap v1701: 196 pgs: 196 active+clean; 583 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-03-21T13:25:10.141 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:25:09 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:25:10.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:10 smithi155 ceph-mon[33846]: osdmap e785: 8 total, 8 up, 8 in 2024-03-21T13:25:10.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:10 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:25:10.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:10 smithi043 ceph-mon[30062]: osdmap e785: 8 total, 8 up, 8 in 2024-03-21T13:25:10.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:10 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:25:10.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:10 smithi043 ceph-mon[41699]: osdmap e785: 8 total, 8 up, 8 in 2024-03-21T13:25:10.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:10 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:25:11.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:11 smithi155 ceph-mon[33846]: osdmap e786: 8 total, 8 up, 8 in 2024-03-21T13:25:11.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:11 smithi155 ceph-mon[33846]: pgmap v1704: 164 pgs: 164 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:25:11.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:11 smithi043 ceph-mon[30062]: osdmap e786: 8 total, 8 up, 8 in 2024-03-21T13:25:11.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:11 smithi043 ceph-mon[30062]: pgmap v1704: 164 pgs: 164 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:25:11.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:11 smithi043 ceph-mon[41699]: osdmap e786: 8 total, 8 up, 8 in 2024-03-21T13:25:11.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:11 smithi043 ceph-mon[41699]: pgmap v1704: 164 pgs: 164 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:25:12.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:12 smithi155 ceph-mon[33846]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:25:12.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:12 smithi155 ceph-mon[33846]: osdmap e787: 8 total, 8 up, 8 in 2024-03-21T13:25:12.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:12 smithi043 ceph-mon[30062]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:25:12.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:12 smithi043 ceph-mon[30062]: osdmap e787: 8 total, 8 up, 8 in 2024-03-21T13:25:12.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:12 smithi043 ceph-mon[41699]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:25:12.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:12 smithi043 ceph-mon[41699]: osdmap e787: 8 total, 8 up, 8 in 2024-03-21T13:25:13.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:13 smithi155 ceph-mon[33846]: osdmap e788: 8 total, 8 up, 8 in 2024-03-21T13:25:13.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:13 smithi155 ceph-mon[33846]: pgmap v1707: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:25:13.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:13 smithi043 ceph-mon[30062]: osdmap e788: 8 total, 8 up, 8 in 2024-03-21T13:25:13.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:13 smithi043 ceph-mon[30062]: pgmap v1707: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:25:13.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:13 smithi043 ceph-mon[41699]: osdmap e788: 8 total, 8 up, 8 in 2024-03-21T13:25:13.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:13 smithi043 ceph-mon[41699]: pgmap v1707: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 840 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:25:14.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:14 smithi155 ceph-mon[33846]: osdmap e789: 8 total, 8 up, 8 in 2024-03-21T13:25:14.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:14 smithi043 ceph-mon[30062]: osdmap e789: 8 total, 8 up, 8 in 2024-03-21T13:25:14.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:14 smithi043 ceph-mon[41699]: osdmap e789: 8 total, 8 up, 8 in 2024-03-21T13:25:15.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:15 smithi155 ceph-mon[33846]: osdmap e790: 8 total, 8 up, 8 in 2024-03-21T13:25:15.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:15 smithi155 ceph-mon[33846]: pgmap v1710: 196 pgs: 196 active+clean; 584 KiB data, 841 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:25:15.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:15 smithi043 ceph-mon[30062]: osdmap e790: 8 total, 8 up, 8 in 2024-03-21T13:25:15.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:15 smithi043 ceph-mon[30062]: pgmap v1710: 196 pgs: 196 active+clean; 584 KiB data, 841 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:25:15.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:15 smithi043 ceph-mon[41699]: osdmap e790: 8 total, 8 up, 8 in 2024-03-21T13:25:15.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:15 smithi043 ceph-mon[41699]: pgmap v1710: 196 pgs: 196 active+clean; 584 KiB data, 841 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:25:16.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:16 smithi155 ceph-mon[33846]: osdmap e791: 8 total, 8 up, 8 in 2024-03-21T13:25:16.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:16 smithi043 ceph-mon[30062]: osdmap e791: 8 total, 8 up, 8 in 2024-03-21T13:25:16.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:16 smithi043 ceph-mon[41699]: osdmap e791: 8 total, 8 up, 8 in 2024-03-21T13:25:17.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:17 smithi155 ceph-mon[33846]: osdmap e792: 8 total, 8 up, 8 in 2024-03-21T13:25:17.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:17 smithi155 ceph-mon[33846]: pgmap v1713: 196 pgs: 196 active+clean; 584 KiB data, 841 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:25:17.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:17 smithi043 ceph-mon[30062]: osdmap e792: 8 total, 8 up, 8 in 2024-03-21T13:25:17.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:17 smithi043 ceph-mon[30062]: pgmap v1713: 196 pgs: 196 active+clean; 584 KiB data, 841 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:25:17.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:17 smithi043 ceph-mon[41699]: osdmap e792: 8 total, 8 up, 8 in 2024-03-21T13:25:17.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:17 smithi043 ceph-mon[41699]: pgmap v1713: 196 pgs: 196 active+clean; 584 KiB data, 841 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-21T13:25:18.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:18 smithi155 ceph-mon[33846]: osdmap e793: 8 total, 8 up, 8 in 2024-03-21T13:25:18.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:18 smithi155 ceph-mon[33846]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:25:18.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:25:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:25:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:25:18.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:18 smithi043 ceph-mon[41699]: osdmap e793: 8 total, 8 up, 8 in 2024-03-21T13:25:18.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:18 smithi043 ceph-mon[41699]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:25:18.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:18 smithi043 ceph-mon[30062]: osdmap e793: 8 total, 8 up, 8 in 2024-03-21T13:25:18.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:18 smithi043 ceph-mon[30062]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:25:19.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:19 smithi155 ceph-mon[33846]: osdmap e794: 8 total, 8 up, 8 in 2024-03-21T13:25:19.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:19 smithi155 ceph-mon[33846]: pgmap v1716: 164 pgs: 164 active+clean; 582 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-21T13:25:19.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:19 smithi043 ceph-mon[30062]: osdmap e794: 8 total, 8 up, 8 in 2024-03-21T13:25:19.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:19 smithi043 ceph-mon[30062]: pgmap v1716: 164 pgs: 164 active+clean; 582 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-21T13:25:19.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:19 smithi043 ceph-mon[41699]: osdmap e794: 8 total, 8 up, 8 in 2024-03-21T13:25:19.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:19 smithi043 ceph-mon[41699]: pgmap v1716: 164 pgs: 164 active+clean; 582 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-21T13:25:20.200 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:25:19 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:25:20.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:20 smithi155 ceph-mon[33846]: osdmap e795: 8 total, 8 up, 8 in 2024-03-21T13:25:20.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:20 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:25:20.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:20 smithi043 ceph-mon[30062]: osdmap e795: 8 total, 8 up, 8 in 2024-03-21T13:25:20.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:20 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:25:20.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:20 smithi043 ceph-mon[41699]: osdmap e795: 8 total, 8 up, 8 in 2024-03-21T13:25:20.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:20 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:25:21.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:21 smithi043 ceph-mon[30062]: osdmap e796: 8 total, 8 up, 8 in 2024-03-21T13:25:21.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:21 smithi043 ceph-mon[30062]: pgmap v1719: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-21T13:25:21.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:21 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:25:21.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:21 smithi043 ceph-mon[41699]: osdmap e796: 8 total, 8 up, 8 in 2024-03-21T13:25:21.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:21 smithi043 ceph-mon[41699]: pgmap v1719: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-21T13:25:21.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:21 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:25:21.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:21 smithi155 ceph-mon[33846]: osdmap e796: 8 total, 8 up, 8 in 2024-03-21T13:25:21.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:21 smithi155 ceph-mon[33846]: pgmap v1719: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 845 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-21T13:25:21.708 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:21 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:25:22.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:22 smithi043 ceph-mon[30062]: osdmap e797: 8 total, 8 up, 8 in 2024-03-21T13:25:22.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:22 smithi043 ceph-mon[41699]: osdmap e797: 8 total, 8 up, 8 in 2024-03-21T13:25:22.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:22 smithi155 ceph-mon[33846]: osdmap e797: 8 total, 8 up, 8 in 2024-03-21T13:25:23.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:23 smithi043 ceph-mon[30062]: osdmap e798: 8 total, 8 up, 8 in 2024-03-21T13:25:23.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:23 smithi043 ceph-mon[30062]: pgmap v1722: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 845 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:25:23.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:23 smithi043 ceph-mon[41699]: osdmap e798: 8 total, 8 up, 8 in 2024-03-21T13:25:23.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:23 smithi043 ceph-mon[41699]: pgmap v1722: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 845 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:25:23.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:23 smithi155 ceph-mon[33846]: osdmap e798: 8 total, 8 up, 8 in 2024-03-21T13:25:23.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:23 smithi155 ceph-mon[33846]: pgmap v1722: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 845 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:25:24.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:24 smithi155 ceph-mon[33846]: osdmap e799: 8 total, 8 up, 8 in 2024-03-21T13:25:24.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:24 smithi043 ceph-mon[30062]: osdmap e799: 8 total, 8 up, 8 in 2024-03-21T13:25:24.733 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:24 smithi043 ceph-mon[41699]: osdmap e799: 8 total, 8 up, 8 in 2024-03-21T13:25:25.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:25 smithi155 ceph-mon[33846]: osdmap e800: 8 total, 8 up, 8 in 2024-03-21T13:25:25.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:25 smithi155 ceph-mon[33846]: pgmap v1725: 164 pgs: 164 active+clean; 582 KiB data, 830 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:25:25.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:25 smithi043 ceph-mon[30062]: osdmap e800: 8 total, 8 up, 8 in 2024-03-21T13:25:25.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:25 smithi043 ceph-mon[30062]: pgmap v1725: 164 pgs: 164 active+clean; 582 KiB data, 830 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:25:25.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:25 smithi043 ceph-mon[41699]: osdmap e800: 8 total, 8 up, 8 in 2024-03-21T13:25:25.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:25 smithi043 ceph-mon[41699]: pgmap v1725: 164 pgs: 164 active+clean; 582 KiB data, 830 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:25:26.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:26 smithi155 ceph-mon[33846]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:25:26.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:26 smithi155 ceph-mon[33846]: osdmap e801: 8 total, 8 up, 8 in 2024-03-21T13:25:26.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:26 smithi043 ceph-mon[30062]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:25:26.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:26 smithi043 ceph-mon[30062]: osdmap e801: 8 total, 8 up, 8 in 2024-03-21T13:25:26.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:26 smithi043 ceph-mon[41699]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:25:26.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:26 smithi043 ceph-mon[41699]: osdmap e801: 8 total, 8 up, 8 in 2024-03-21T13:25:27.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:27 smithi155 ceph-mon[33846]: osdmap e802: 8 total, 8 up, 8 in 2024-03-21T13:25:27.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:27 smithi155 ceph-mon[33846]: pgmap v1728: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 830 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:25:27.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:27 smithi043 ceph-mon[30062]: osdmap e802: 8 total, 8 up, 8 in 2024-03-21T13:25:27.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:27 smithi043 ceph-mon[30062]: pgmap v1728: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 830 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:25:27.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:27 smithi043 ceph-mon[41699]: osdmap e802: 8 total, 8 up, 8 in 2024-03-21T13:25:27.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:27 smithi043 ceph-mon[41699]: pgmap v1728: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 830 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:25:28.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:28 smithi043 ceph-mon[30062]: osdmap e803: 8 total, 8 up, 8 in 2024-03-21T13:25:28.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:28 smithi043 ceph-mon[41699]: osdmap e803: 8 total, 8 up, 8 in 2024-03-21T13:25:28.484 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:25:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:25:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:25:28.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:28 smithi155 ceph-mon[33846]: osdmap e803: 8 total, 8 up, 8 in 2024-03-21T13:25:29.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:29 smithi155 ceph-mon[33846]: osdmap e804: 8 total, 8 up, 8 in 2024-03-21T13:25:29.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:29 smithi155 ceph-mon[33846]: pgmap v1731: 164 pgs: 164 active+clean; 582 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:25:29.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:29 smithi043 ceph-mon[30062]: osdmap e804: 8 total, 8 up, 8 in 2024-03-21T13:25:29.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:29 smithi043 ceph-mon[30062]: pgmap v1731: 164 pgs: 164 active+clean; 582 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:25:29.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:29 smithi043 ceph-mon[41699]: osdmap e804: 8 total, 8 up, 8 in 2024-03-21T13:25:29.735 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:29 smithi043 ceph-mon[41699]: pgmap v1731: 164 pgs: 164 active+clean; 582 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:25:30.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:25:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:25:30.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:30 smithi155 ceph-mon[33846]: osdmap e805: 8 total, 8 up, 8 in 2024-03-21T13:25:30.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:30 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:25:30.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:30 smithi043 ceph-mon[30062]: osdmap e805: 8 total, 8 up, 8 in 2024-03-21T13:25:30.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:30 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:25:30.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:30 smithi043 ceph-mon[41699]: osdmap e805: 8 total, 8 up, 8 in 2024-03-21T13:25:30.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:30 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:25:31.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:31 smithi155 ceph-mon[33846]: osdmap e806: 8 total, 8 up, 8 in 2024-03-21T13:25:31.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:31 smithi155 ceph-mon[33846]: pgmap v1734: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:25:31.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:31 smithi043 ceph-mon[30062]: osdmap e806: 8 total, 8 up, 8 in 2024-03-21T13:25:31.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:31 smithi043 ceph-mon[30062]: pgmap v1734: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:25:31.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:31 smithi043 ceph-mon[41699]: osdmap e806: 8 total, 8 up, 8 in 2024-03-21T13:25:31.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:31 smithi043 ceph-mon[41699]: pgmap v1734: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:25:32.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:32 smithi155 ceph-mon[33846]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:25:32.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:32 smithi155 ceph-mon[33846]: osdmap e807: 8 total, 8 up, 8 in 2024-03-21T13:25:32.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:32 smithi043 ceph-mon[30062]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:25:32.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:32 smithi043 ceph-mon[30062]: osdmap e807: 8 total, 8 up, 8 in 2024-03-21T13:25:32.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:32 smithi043 ceph-mon[41699]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:25:32.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:32 smithi043 ceph-mon[41699]: osdmap e807: 8 total, 8 up, 8 in 2024-03-21T13:25:33.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:33 smithi155 ceph-mon[33846]: osdmap e808: 8 total, 8 up, 8 in 2024-03-21T13:25:33.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:33 smithi155 ceph-mon[33846]: pgmap v1737: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 883 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:25:33.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:33 smithi043 ceph-mon[30062]: osdmap e808: 8 total, 8 up, 8 in 2024-03-21T13:25:33.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:33 smithi043 ceph-mon[30062]: pgmap v1737: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 883 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:25:33.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:33 smithi043 ceph-mon[41699]: osdmap e808: 8 total, 8 up, 8 in 2024-03-21T13:25:33.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:33 smithi043 ceph-mon[41699]: pgmap v1737: 196 pgs: 32 unknown, 164 active+clean; 582 KiB data, 883 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:25:35.909 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:35 smithi155 ceph-mon[33846]: pgmap v1738: 196 pgs: 196 active+clean; 583 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 962 B/s rd, 192 B/s wr, 1 op/s 2024-03-21T13:25:35.910 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:25:35.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:35 smithi043 ceph-mon[41699]: pgmap v1738: 196 pgs: 196 active+clean; 583 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 962 B/s rd, 192 B/s wr, 1 op/s 2024-03-21T13:25:35.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:25:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:35 smithi043 ceph-mon[30062]: pgmap v1738: 196 pgs: 196 active+clean; 583 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 962 B/s rd, 192 B/s wr, 1 op/s 2024-03-21T13:25:35.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:25:36.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:36 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:25:36.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:36 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:25:36.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:36 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:25:37.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:37 smithi155 ceph-mon[33846]: pgmap v1739: 196 pgs: 196 active+clean; 583 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 810 B/s rd, 162 B/s wr, 0 op/s 2024-03-21T13:25:37.908 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:37 smithi043 ceph-mon[30062]: pgmap v1739: 196 pgs: 196 active+clean; 583 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 810 B/s rd, 162 B/s wr, 0 op/s 2024-03-21T13:25:37.908 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:37 smithi043 ceph-mon[41699]: pgmap v1739: 196 pgs: 196 active+clean; 583 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 810 B/s rd, 162 B/s wr, 0 op/s 2024-03-21T13:25:38.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:25:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:25:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:25:39.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:39 smithi155 ceph-mon[33846]: osdmap e809: 8 total, 8 up, 8 in 2024-03-21T13:25:39.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:39 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:25:39.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:39 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:25:39.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:39 smithi155 ceph-mon[33846]: pgmap v1741: 196 pgs: 196 active+clean; 583 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 140 B/s wr, 1 op/s 2024-03-21T13:25:39.483 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:39 smithi043 ceph-mon[41699]: osdmap e809: 8 total, 8 up, 8 in 2024-03-21T13:25:39.483 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:39 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:25:39.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:39 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:25:39.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:39 smithi043 ceph-mon[41699]: pgmap v1741: 196 pgs: 196 active+clean; 583 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 140 B/s wr, 1 op/s 2024-03-21T13:25:39.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:39 smithi043 ceph-mon[30062]: osdmap e809: 8 total, 8 up, 8 in 2024-03-21T13:25:39.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:39 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:25:39.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:39 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:25:39.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:39 smithi043 ceph-mon[30062]: pgmap v1741: 196 pgs: 196 active+clean; 583 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 140 B/s wr, 1 op/s 2024-03-21T13:25:40.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:25:39 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:25:40.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:40 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:25:40.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:40 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:25:40.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:40 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:25:40.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:40 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:40.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:40 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:25:40.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:40 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:25:40.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:40 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:25:40.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:40 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:25:40.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:40 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:25:40.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:40 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:40.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:40 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:25:40.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:40 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:25:40.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:40 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:25:40.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:40 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:25:40.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:40 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:25:40.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:40 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:25:40.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:40 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:25:40.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:40 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:25:41.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:41 smithi155 ceph-mon[33846]: pgmap v1742: 196 pgs: 196 active+clean; 583 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 123 B/s wr, 1 op/s 2024-03-21T13:25:41.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:41 smithi043 ceph-mon[30062]: pgmap v1742: 196 pgs: 196 active+clean; 583 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 123 B/s wr, 1 op/s 2024-03-21T13:25:41.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:41 smithi043 ceph-mon[41699]: pgmap v1742: 196 pgs: 196 active+clean; 583 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 123 B/s wr, 1 op/s 2024-03-21T13:25:43.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:43 smithi155 ceph-mon[33846]: pgmap v1743: 196 pgs: 196 active+clean; 583 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-03-21T13:25:43.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:43 smithi043 ceph-mon[30062]: pgmap v1743: 196 pgs: 196 active+clean; 583 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-03-21T13:25:43.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:43 smithi043 ceph-mon[41699]: pgmap v1743: 196 pgs: 196 active+clean; 583 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 102 B/s wr, 1 op/s 2024-03-21T13:25:45.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:45 smithi155 ceph-mon[33846]: pgmap v1744: 196 pgs: 196 active+clean; 583 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:25:45.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:45 smithi043 ceph-mon[30062]: pgmap v1744: 196 pgs: 196 active+clean; 583 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:25:45.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:45 smithi043 ceph-mon[41699]: pgmap v1744: 196 pgs: 196 active+clean; 583 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:25:47.666 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: Running main() from gmock_main.cc 2024-03-21T13:25:47.667 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [==========] Running 11 tests from 2 test suites. 2024-03-21T13:25:47.667 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [----------] Global test environment set-up. 2024-03-21T13:25:47.667 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots 2024-03-21T13:25:47.667 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapList 2024-03-21T13:25:47.667 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapList (1811746 ms) 2024-03-21T13:25:47.667 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapRemove 2024-03-21T13:25:47.667 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapRemove (5033 ms) 2024-03-21T13:25:47.667 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSnapshots.Rollback 2024-03-21T13:25:47.667 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSnapshots.Rollback (4048 ms) 2024-03-21T13:25:47.667 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapGetName 2024-03-21T13:25:47.667 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapGetName (5088 ms) 2024-03-21T13:25:47.667 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapCreateRemove 2024-03-21T13:25:47.668 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapCreateRemove (7101 ms) 2024-03-21T13:25:47.668 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots (1833016 ms total) 2024-03-21T13:25:47.668 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: 2024-03-21T13:25:47.668 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps 2024-03-21T13:25:47.668 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Snap 2024-03-21T13:25:47.668 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Snap (5060 ms) 2024-03-21T13:25:47.668 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Rollback 2024-03-21T13:25:47.668 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Rollback (6031 ms) 2024-03-21T13:25:47.668 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.SnapOverlap 2024-03-21T13:25:47.668 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.SnapOverlap (8049 ms) 2024-03-21T13:25:47.668 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Bug11677 2024-03-21T13:25:47.668 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Bug11677 (6036 ms) 2024-03-21T13:25:47.668 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.OrderSnap 2024-03-21T13:25:47.669 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.OrderSnap (4026 ms) 2024-03-21T13:25:47.669 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.ReusePurgedSnap 2024-03-21T13:25:47.669 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: Deleting snap 3 in pool ReusePurgedSnapsmithi043-82997-11. 2024-03-21T13:25:47.669 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: Waiting for snaps to purge. 2024-03-21T13:25:47.669 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.ReusePurgedSnap (19399 ms) 2024-03-21T13:25:47.669 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps (48601 ms total) 2024-03-21T13:25:47.669 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: 2024-03-21T13:25:47.669 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [----------] Global test environment tear-down 2024-03-21T13:25:47.669 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [==========] 11 tests from 2 test suites ran. (1881617 ms total) 2024-03-21T13:25:47.669 INFO:tasks.workunit.client.0.smithi043.stdout: snapshots: [ PASSED ] 11 tests. 2024-03-21T13:25:47.670 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:47.670 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82087 2024-03-21T13:25:47.670 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82087 2024-03-21T13:25:47.670 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:47.670 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81575 2024-03-21T13:25:47.670 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81575 2024-03-21T13:25:47.670 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:47.671 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81639 2024-03-21T13:25:47.671 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81639 2024-03-21T13:25:47.671 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:47.671 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82042 2024-03-21T13:25:47.671 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82042 2024-03-21T13:25:47.671 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:47.671 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=81678 2024-03-21T13:25:47.671 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 81678 2024-03-21T13:25:47.671 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:47.671 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82110 2024-03-21T13:25:47.671 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82110 2024-03-21T13:25:47.671 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:47.671 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82164 2024-03-21T13:25:47.671 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82164 2024-03-21T13:25:47.672 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:47.672 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82663 2024-03-21T13:25:47.672 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82663 2024-03-21T13:25:47.672 INFO:tasks.workunit.client.0.smithi043.stderr:+ for t in "${!pids[@]}" 2024-03-21T13:25:47.672 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=82949 2024-03-21T13:25:47.672 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 82949 2024-03-21T13:25:47.672 INFO:tasks.workunit.client.0.smithi043.stderr:+ exit 0 2024-03-21T13:25:47.672 INFO:tasks.workunit.client.0.smithi043.stderr:+ cleanup 2024-03-21T13:25:47.672 INFO:tasks.workunit.client.0.smithi043.stderr:+ pkill -P 81569 2024-03-21T13:25:47.693 INFO:tasks.workunit.client.0.smithi043.stderr:+ true 2024-03-21T13:25:47.693 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-21T13:25:47.694 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-21T13:25:47.727 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-03-21T13:25:47.728 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-03-21T13:25:47.728 DEBUG:teuthology.orchestra.run.smithi043:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=960fbbe4645c2becc8084669be194c95183c7753 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-03-21T13:25:47.788 INFO:tasks.workunit.client.0.smithi043.stderr:++ uuidgen 2024-03-21T13:25:47.795 INFO:tasks.workunit.client.0.smithi043.stderr:+ p=dfa6c75f-2043-4860-bf82-1f19a1404959 2024-03-21T13:25:47.795 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool create dfa6c75f-2043-4860-bf82-1f19a1404959 12 2024-03-21T13:25:47.881 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.881+0000 7f3191415640 1 -- 172.21.15.43:0/213772082 >> v1:172.21.15.43:6790/0 conn(0x7f318c102750 legacy=0x7f318c102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:47.881 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.881+0000 7f3191415640 1 -- 172.21.15.43:0/213772082 shutdown_connections 2024-03-21T13:25:47.881 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.881+0000 7f3191415640 1 -- 172.21.15.43:0/213772082 >> 172.21.15.43:0/213772082 conn(0x7f318c0770b0 msgr2=0x7f318c0fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:25:47.881 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.881+0000 7f3191415640 1 -- 172.21.15.43:0/213772082 shutdown_connections 2024-03-21T13:25:47.881 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.881+0000 7f3191415640 1 -- 172.21.15.43:0/213772082 wait complete. 2024-03-21T13:25:47.881 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.881+0000 7f3191415640 1 Processor -- start 2024-03-21T13:25:47.881 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.882+0000 7f3191415640 1 -- start start 2024-03-21T13:25:47.884 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.882+0000 7f3191415640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f318c10a0a0 con 0x7f318c10a550 2024-03-21T13:25:47.884 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.882+0000 7f3191415640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f318c10a2d0 con 0x7f318c10e060 2024-03-21T13:25:47.884 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.882+0000 7f3191415640 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f318c1a8b30 con 0x7f318c102750 2024-03-21T13:25:47.884 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.882+0000 7f318affd640 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7f318c102750 0x7f318c106f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:43482/0 (socket says 172.21.15.43:43482) 2024-03-21T13:25:47.884 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.882+0000 7f318affd640 1 -- 172.21.15.43:0/2973625811 learned_addr learned my addr 172.21.15.43:0/2973625811 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T13:25:47.884 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.882+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2340125783 0 0) 0x7f318c1a8b30 con 0x7f318c102750 2024-03-21T13:25:47.884 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.882+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3178003160 con 0x7f318c102750 2024-03-21T13:25:47.884 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.882+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2766173653 0 0) 0x7f318c10a0a0 con 0x7f318c10a550 2024-03-21T13:25:47.884 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.882+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f318c1a8b30 con 0x7f318c10a550 2024-03-21T13:25:47.885 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.882+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3910533795 0 0) 0x7f318c10a2d0 con 0x7f318c10e060 2024-03-21T13:25:47.885 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.882+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f318c10a0a0 con 0x7f318c10e060 2024-03-21T13:25:47.885 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.882+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2191955268 0 0) 0x7f3178003160 con 0x7f318c102750 2024-03-21T13:25:47.885 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.882+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f318c10a2d0 con 0x7f318c102750 2024-03-21T13:25:47.885 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.882+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f3184004530 con 0x7f318c102750 2024-03-21T13:25:47.885 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.882+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 607067794 0 0) 0x7f318c1a8b30 con 0x7f318c10a550 2024-03-21T13:25:47.885 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.882+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3178003160 con 0x7f318c10a550 2024-03-21T13:25:47.885 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.882+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f3174001830 con 0x7f318c10a550 2024-03-21T13:25:47.885 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.883+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2745157140 0 0) 0x7f318c10a0a0 con 0x7f318c10e060 2024-03-21T13:25:47.885 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.883+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f318c1a8b30 con 0x7f318c10e060 2024-03-21T13:25:47.885 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.883+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f317c001470 con 0x7f318c10e060 2024-03-21T13:25:47.886 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.883+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 <== mon.2 v1:172.21.15.43:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 341621601 0 0) 0x7f318c10a2d0 con 0x7f318c102750 2024-03-21T13:25:47.886 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.883+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 >> v1:172.21.15.155:6789/0 conn(0x7f318c10e060 legacy=0x7f318c1a53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:47.886 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.883+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 >> v1:172.21.15.43:6789/0 conn(0x7f318c10a550 legacy=0x7f318c107630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:47.886 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.883+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 --> v1:172.21.15.43:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f318c1a9d50 con 0x7f318c102750 2024-03-21T13:25:47.886 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.883+0000 7f3191415640 1 -- 172.21.15.43:0/2973625811 --> v1:172.21.15.43:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f318c1a8d60 con 0x7f318c102750 2024-03-21T13:25:47.886 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.883+0000 7f3191415640 1 -- 172.21.15.43:0/2973625811 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f318c1a9230 con 0x7f318c102750 2024-03-21T13:25:47.886 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.883+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 <== mon.2 v1:172.21.15.43:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f3184001b70 con 0x7f318c102750 2024-03-21T13:25:47.886 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.883+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 <== mon.2 v1:172.21.15.43:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f3184004dc0 con 0x7f318c102750 2024-03-21T13:25:47.886 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.884+0000 7f3191415640 1 -- 172.21.15.43:0/2973625811 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f316c005080 con 0x7f318c102750 2024-03-21T13:25:47.886 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.884+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 <== mon.2 v1:172.21.15.43:6790/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7f3184005040 con 0x7f318c102750 2024-03-21T13:25:47.886 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.885+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 <== mon.2 v1:172.21.15.43:6790/0 8 ==== osd_map(810..810 src has 251..810) v4 ==== 7410+0+0 (unknown 1520732779 0 0) 0x7f3184089ce0 con 0x7f318c102750 2024-03-21T13:25:47.889 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:47.889+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 <== mon.2 v1:172.21.15.43:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f3184056e30 con 0x7f318c102750 2024-03-21T13:25:47.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:47 smithi155 ceph-mon[33846]: pgmap v1745: 196 pgs: 196 active+clean; 583 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:25:47.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:47 smithi043 ceph-mon[30062]: pgmap v1745: 196 pgs: 196 active+clean; 583 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:25:47.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:47 smithi043 ceph-mon[41699]: pgmap v1745: 196 pgs: 196 active+clean; 583 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:25:48.072 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.073+0000 7f3191415640 1 -- 172.21.15.43:0/2973625811 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pg_num": 12} v 0) v1 -- 0x7f316c005370 con 0x7f318c102750 2024-03-21T13:25:48.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:25:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:25:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:25:48.672 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.673+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 <== mon.2 v1:172.21.15.43:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pg_num": 12}]=0 pool 'dfa6c75f-2043-4860-bf82-1f19a1404959' created v811) v1 ==== 176+0+0 (unknown 1363793666 0 0) 0x7f318405a880 con 0x7f318c102750 2024-03-21T13:25:48.789 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.789+0000 7f3191415640 1 -- 172.21.15.43:0/2973625811 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pg_num": 12} v 0) v1 -- 0x7f316c001170 con 0x7f318c102750 2024-03-21T13:25:48.790 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.790+0000 7f3183fff640 1 -- 172.21.15.43:0/2973625811 <== mon.2 v1:172.21.15.43:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pg_num": 12}]=0 pool 'dfa6c75f-2043-4860-bf82-1f19a1404959' already exists v811) v1 ==== 183+0+0 (unknown 253495154 0 0) 0x7f316c001170 con 0x7f318c102750 2024-03-21T13:25:48.790 INFO:tasks.workunit.client.0.smithi043.stderr:pool 'dfa6c75f-2043-4860-bf82-1f19a1404959' already exists 2024-03-21T13:25:48.792 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.792+0000 7f3191415640 1 -- 172.21.15.43:0/2973625811 >> v1:172.21.15.43:6800/4199387216 conn(0x7f317806f050 legacy=0x7f3178071510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:48.792 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.792+0000 7f3191415640 1 -- 172.21.15.43:0/2973625811 >> v1:172.21.15.43:6790/0 conn(0x7f318c102750 legacy=0x7f318c106f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:48.792 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.792+0000 7f3191415640 1 -- 172.21.15.43:0/2973625811 shutdown_connections 2024-03-21T13:25:48.792 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.792+0000 7f3191415640 1 -- 172.21.15.43:0/2973625811 >> 172.21.15.43:0/2973625811 conn(0x7f318c0770b0 msgr2=0x7f318c10f2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:25:48.792 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.792+0000 7f3191415640 1 -- 172.21.15.43:0/2973625811 shutdown_connections 2024-03-21T13:25:48.792 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.792+0000 7f3191415640 1 -- 172.21.15.43:0/2973625811 wait complete. 2024-03-21T13:25:48.801 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota dfa6c75f-2043-4860-bf82-1f19a1404959 max_objects 10 2024-03-21T13:25:48.885 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.885+0000 7f8dc5ab5640 1 -- 172.21.15.43:0/4013422794 >> v1:172.21.15.43:6789/0 conn(0x7f8dc010df70 legacy=0x7f8dc0110430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:48.885 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.885+0000 7f8dc5ab5640 1 -- 172.21.15.43:0/4013422794 shutdown_connections 2024-03-21T13:25:48.885 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.885+0000 7f8dc5ab5640 1 -- 172.21.15.43:0/4013422794 >> 172.21.15.43:0/4013422794 conn(0x7f8dc00fcfe0 msgr2=0x7f8dc00ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:25:48.885 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.885+0000 7f8dc5ab5640 1 -- 172.21.15.43:0/4013422794 shutdown_connections 2024-03-21T13:25:48.885 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.885+0000 7f8dc5ab5640 1 -- 172.21.15.43:0/4013422794 wait complete. 2024-03-21T13:25:48.886 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.885+0000 7f8dc5ab5640 1 Processor -- start 2024-03-21T13:25:48.886 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.886+0000 7f8dc5ab5640 1 -- start start 2024-03-21T13:25:48.888 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.886+0000 7f8dc5ab5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8dc01a65a0 con 0x7f8dc01075d0 2024-03-21T13:25:48.889 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.886+0000 7f8dc5ab5640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8dc01a77e0 con 0x7f8dc010a370 2024-03-21T13:25:48.889 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.886+0000 7f8dc5ab5640 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8dc01a8a20 con 0x7f8dc010df70 2024-03-21T13:25:48.889 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.886+0000 7f8dbf7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8dc01075d0 0x7f8dc0106960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59322/0 (socket says 172.21.15.43:59322) 2024-03-21T13:25:48.889 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.886+0000 7f8dbf7fe640 1 -- 172.21.15.43:0/115109259 learned_addr learned my addr 172.21.15.43:0/115109259 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T13:25:48.889 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.886+0000 7f8dbffff640 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7f8dc010df70 0x7f8dc01a4ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:43490/0 (socket says 172.21.15.43:43490) 2024-03-21T13:25:48.889 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.886+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773813736 0 0) 0x7f8dc01a65a0 con 0x7f8dc01075d0 2024-03-21T13:25:48.889 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.886+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8da8003520 con 0x7f8dc01075d0 2024-03-21T13:25:48.889 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.886+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4138294989 0 0) 0x7f8dc01a8a20 con 0x7f8dc010df70 2024-03-21T13:25:48.889 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.886+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dc01a65a0 con 0x7f8dc010df70 2024-03-21T13:25:48.889 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.886+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2819077893 0 0) 0x7f8dc01a77e0 con 0x7f8dc010a370 2024-03-21T13:25:48.889 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.886+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dc01a8a20 con 0x7f8dc010a370 2024-03-21T13:25:48.889 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.886+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4131055480 0 0) 0x7f8dc01a65a0 con 0x7f8dc010df70 2024-03-21T13:25:48.889 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.886+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8dc01a77e0 con 0x7f8dc010df70 2024-03-21T13:25:48.890 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.886+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2958212147 0 0) 0x7f8da8003520 con 0x7f8dc01075d0 2024-03-21T13:25:48.890 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.887+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8dc01a65a0 con 0x7f8dc01075d0 2024-03-21T13:25:48.890 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.887+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f8db8004660 con 0x7f8dc010df70 2024-03-21T13:25:48.890 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.887+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f8db00018b0 con 0x7f8dc01075d0 2024-03-21T13:25:48.890 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.887+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 368169033 0 0) 0x7f8dc01a8a20 con 0x7f8dc010a370 2024-03-21T13:25:48.890 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.887+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8da8003520 con 0x7f8dc010a370 2024-03-21T13:25:48.890 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.887+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f8db4001890 con 0x7f8dc010a370 2024-03-21T13:25:48.890 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.887+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 <== mon.2 v1:172.21.15.43:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3252098928 0 0) 0x7f8dc01a77e0 con 0x7f8dc010df70 2024-03-21T13:25:48.890 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.887+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 >> v1:172.21.15.155:6789/0 conn(0x7f8dc010a370 legacy=0x7f8dc01a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:48.890 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.887+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 >> v1:172.21.15.43:6789/0 conn(0x7f8dc01075d0 legacy=0x7f8dc0106960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:48.890 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.887+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 --> v1:172.21.15.43:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc01a9c60 con 0x7f8dc010df70 2024-03-21T13:25:48.890 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.887+0000 7f8dc5ab5640 1 -- 172.21.15.43:0/115109259 --> v1:172.21.15.43:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc01a67d0 con 0x7f8dc010df70 2024-03-21T13:25:48.890 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.887+0000 7f8dc5ab5640 1 -- 172.21.15.43:0/115109259 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc01a6ca0 con 0x7f8dc010df70 2024-03-21T13:25:48.891 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.887+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 <== mon.2 v1:172.21.15.43:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f8db80049a0 con 0x7f8dc010df70 2024-03-21T13:25:48.891 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.887+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 <== mon.2 v1:172.21.15.43:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f8db8004e50 con 0x7f8dc010df70 2024-03-21T13:25:48.891 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.888+0000 7f8dc5ab5640 1 -- 172.21.15.43:0/115109259 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dac005080 con 0x7f8dc010df70 2024-03-21T13:25:48.891 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.888+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 <== mon.2 v1:172.21.15.43:6790/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7f8db8004e30 con 0x7f8dc010df70 2024-03-21T13:25:48.891 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.889+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 <== mon.2 v1:172.21.15.43:6790/0 8 ==== osd_map(811..811 src has 251..811) v4 ==== 7785+0+0 (unknown 2247394942 0 0) 0x7f8db8089fe0 con 0x7f8dc010df70 2024-03-21T13:25:48.892 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:48.893+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 <== mon.2 v1:172.21.15.43:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f8db8056fc0 con 0x7f8dc010df70 2024-03-21T13:25:48.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:48 smithi155 ceph-mon[33846]: osdmap e810: 8 total, 8 up, 8 in 2024-03-21T13:25:48.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:48 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2973625811' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pg_num": 12}]: dispatch 2024-03-21T13:25:48.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:48 smithi155 ceph-mon[33846]: from='client.58694 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pg_num": 12}]: dispatch 2024-03-21T13:25:48.959 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:48 smithi043 ceph-mon[30062]: osdmap e810: 8 total, 8 up, 8 in 2024-03-21T13:25:48.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:48 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2973625811' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pg_num": 12}]: dispatch 2024-03-21T13:25:48.960 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:48 smithi043 ceph-mon[30062]: from='client.58694 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pg_num": 12}]: dispatch 2024-03-21T13:25:48.960 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:48 smithi043 ceph-mon[41699]: osdmap e810: 8 total, 8 up, 8 in 2024-03-21T13:25:48.960 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:48 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2973625811' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pg_num": 12}]: dispatch 2024-03-21T13:25:48.960 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:48 smithi043 ceph-mon[41699]: from='client.58694 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pg_num": 12}]: dispatch 2024-03-21T13:25:49.098 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:49.098+0000 7f8dc5ab5640 1 -- 172.21.15.43:0/115109259 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f8dac005370 con 0x7f8dc010df70 2024-03-21T13:25:49.677 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:49.677+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 <== mon.2 v1:172.21.15.43:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool dfa6c75f-2043-4860-bf82-1f19a1404959 v812) v1 ==== 223+0+0 (unknown 3348338656 0 0) 0x7f8db805aa10 con 0x7f8dc010df70 2024-03-21T13:25:49.793 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:49.793+0000 7f8dc5ab5640 1 -- 172.21.15.43:0/115109259 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f8dac001090 con 0x7f8dc010df70 2024-03-21T13:25:49.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:25:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:25:49.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:49 smithi155 ceph-mon[33846]: pgmap v1747: 164 pgs: 164 active+clean; 582 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:25:49.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:49 smithi155 ceph-mon[33846]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:25:49.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:49 smithi155 ceph-mon[33846]: from='client.58694 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pg_num": 12}]': finished 2024-03-21T13:25:49.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:49 smithi155 ceph-mon[33846]: osdmap e811: 8 total, 8 up, 8 in 2024-03-21T13:25:49.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:49 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2973625811' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pg_num": 12}]: dispatch 2024-03-21T13:25:49.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:49 smithi155 ceph-mon[33846]: from='client.58694 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pg_num": 12}]: dispatch 2024-03-21T13:25:49.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:49 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/115109259' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:25:49.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:49 smithi155 ceph-mon[33846]: from='client.58700 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:25:49.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:49 smithi043 ceph-mon[30062]: pgmap v1747: 164 pgs: 164 active+clean; 582 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:25:49.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:49 smithi043 ceph-mon[30062]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:25:49.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:49 smithi043 ceph-mon[30062]: from='client.58694 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pg_num": 12}]': finished 2024-03-21T13:25:49.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:49 smithi043 ceph-mon[30062]: osdmap e811: 8 total, 8 up, 8 in 2024-03-21T13:25:49.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:49 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2973625811' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pg_num": 12}]: dispatch 2024-03-21T13:25:49.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:49 smithi043 ceph-mon[30062]: from='client.58694 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pg_num": 12}]: dispatch 2024-03-21T13:25:49.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:49 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/115109259' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:25:49.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:49 smithi043 ceph-mon[30062]: from='client.58700 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:25:49.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:49 smithi043 ceph-mon[41699]: pgmap v1747: 164 pgs: 164 active+clean; 582 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:25:49.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:49 smithi043 ceph-mon[41699]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:25:49.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:49 smithi043 ceph-mon[41699]: from='client.58694 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pg_num": 12}]': finished 2024-03-21T13:25:49.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:49 smithi043 ceph-mon[41699]: osdmap e811: 8 total, 8 up, 8 in 2024-03-21T13:25:49.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:49 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2973625811' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pg_num": 12}]: dispatch 2024-03-21T13:25:49.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:49 smithi043 ceph-mon[41699]: from='client.58694 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pg_num": 12}]: dispatch 2024-03-21T13:25:49.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:49 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/115109259' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:25:49.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:49 smithi043 ceph-mon[41699]: from='client.58700 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:25:50.684 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.684+0000 7f8dbcff9640 1 -- 172.21.15.43:0/115109259 <== mon.2 v1:172.21.15.43:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool dfa6c75f-2043-4860-bf82-1f19a1404959 v813) v1 ==== 223+0+0 (unknown 2396402320 0 0) 0x7f8db805f2d0 con 0x7f8dc010df70 2024-03-21T13:25:50.684 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_objects = 10 for pool dfa6c75f-2043-4860-bf82-1f19a1404959 2024-03-21T13:25:50.687 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.687+0000 7f8dc5ab5640 1 -- 172.21.15.43:0/115109259 >> v1:172.21.15.43:6800/4199387216 conn(0x7f8da806f120 legacy=0x7f8da80715e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:50.687 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.687+0000 7f8dc5ab5640 1 -- 172.21.15.43:0/115109259 >> v1:172.21.15.43:6790/0 conn(0x7f8dc010df70 legacy=0x7f8dc01a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:50.687 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.687+0000 7f8dc5ab5640 1 -- 172.21.15.43:0/115109259 shutdown_connections 2024-03-21T13:25:50.687 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.687+0000 7f8dc5ab5640 1 -- 172.21.15.43:0/115109259 >> 172.21.15.43:0/115109259 conn(0x7f8dc00fcfe0 msgr2=0x7f8dc00fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:25:50.687 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.688+0000 7f8dc5ab5640 1 -- 172.21.15.43:0/115109259 shutdown_connections 2024-03-21T13:25:50.687 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.688+0000 7f8dc5ab5640 1 -- 172.21.15.43:0/115109259 wait complete. 2024-03-21T13:25:50.699 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool application enable dfa6c75f-2043-4860-bf82-1f19a1404959 rados 2024-03-21T13:25:50.791 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.788+0000 7fe29bc84640 1 -- 172.21.15.43:0/1895597411 >> v1:172.21.15.43:6789/0 conn(0x7fe29410a330 legacy=0x7fe29410c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:50.791 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.788+0000 7fe29bc84640 1 -- 172.21.15.43:0/1895597411 shutdown_connections 2024-03-21T13:25:50.791 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.788+0000 7fe29bc84640 1 -- 172.21.15.43:0/1895597411 >> 172.21.15.43:0/1895597411 conn(0x7fe2940fcfa0 msgr2=0x7fe2940ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:25:50.791 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.788+0000 7fe29bc84640 1 -- 172.21.15.43:0/1895597411 shutdown_connections 2024-03-21T13:25:50.791 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.788+0000 7fe29bc84640 1 -- 172.21.15.43:0/1895597411 wait complete. 2024-03-21T13:25:50.791 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.789+0000 7fe29bc84640 1 Processor -- start 2024-03-21T13:25:50.792 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.789+0000 7fe29bc84640 1 -- start start 2024-03-21T13:25:50.792 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.789+0000 7fe29bc84640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe2941a6510 con 0x7fe29410df30 2024-03-21T13:25:50.792 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.789+0000 7fe29bc84640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe2941a7750 con 0x7fe294107590 2024-03-21T13:25:50.792 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.789+0000 7fe29bc84640 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe2941a8990 con 0x7fe29410a330 2024-03-21T13:25:50.792 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.789+0000 7fe29a221640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe29410df30 0x7fe2941a4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59342/0 (socket says 172.21.15.43:59342) 2024-03-21T13:25:50.792 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.789+0000 7fe29a221640 1 -- 172.21.15.43:0/2241308727 learned_addr learned my addr 172.21.15.43:0/2241308727 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T13:25:50.792 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.789+0000 7fe29921f640 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7fe29410a330 0x7fe2941a13a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:43508/0 (socket says 172.21.15.43:43508) 2024-03-21T13:25:50.792 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.789+0000 7fe28affd640 1 -- 172.21.15.43:0/2241308727 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 389576892 0 0) 0x7fe2941a6510 con 0x7fe29410df30 2024-03-21T13:25:50.792 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.789+0000 7fe28affd640 1 -- 172.21.15.43:0/2241308727 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe280003520 con 0x7fe29410df30 2024-03-21T13:25:50.792 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.790+0000 7fe28affd640 1 -- 172.21.15.43:0/2241308727 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3419519202 0 0) 0x7fe2941a8990 con 0x7fe29410a330 2024-03-21T13:25:50.792 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.790+0000 7fe28affd640 1 -- 172.21.15.43:0/2241308727 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2941a6510 con 0x7fe29410a330 2024-03-21T13:25:50.792 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.790+0000 7fe28affd640 1 -- 172.21.15.43:0/2241308727 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3464059076 0 0) 0x7fe280003520 con 0x7fe29410df30 2024-03-21T13:25:50.792 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.790+0000 7fe28affd640 1 -- 172.21.15.43:0/2241308727 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe2941a8990 con 0x7fe29410df30 2024-03-21T13:25:50.793 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.790+0000 7fe28affd640 1 -- 172.21.15.43:0/2241308727 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fe290001c90 con 0x7fe29410df30 2024-03-21T13:25:50.793 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.790+0000 7fe28affd640 1 -- 172.21.15.43:0/2241308727 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3511802605 0 0) 0x7fe2941a6510 con 0x7fe29410a330 2024-03-21T13:25:50.793 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.790+0000 7fe28affd640 1 -- 172.21.15.43:0/2241308727 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe280003520 con 0x7fe29410a330 2024-03-21T13:25:50.793 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.790+0000 7fe28affd640 1 -- 172.21.15.43:0/2241308727 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2154639799 0 0) 0x7fe2941a7750 con 0x7fe294107590 2024-03-21T13:25:50.793 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.790+0000 7fe28affd640 1 -- 172.21.15.43:0/2241308727 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2941a6510 con 0x7fe294107590 2024-03-21T13:25:50.793 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.790+0000 7fe28affd640 1 -- 172.21.15.43:0/2241308727 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fe284001370 con 0x7fe29410a330 2024-03-21T13:25:50.793 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.790+0000 7fe28affd640 1 -- 172.21.15.43:0/2241308727 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 384878698 0 0) 0x7fe2941a8990 con 0x7fe29410df30 2024-03-21T13:25:50.793 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.790+0000 7fe28affd640 1 -- 172.21.15.43:0/2241308727 >> v1:172.21.15.43:6790/0 conn(0x7fe29410a330 legacy=0x7fe2941a13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:50.793 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.790+0000 7fe28affd640 1 -- 172.21.15.43:0/2241308727 >> v1:172.21.15.155:6789/0 conn(0x7fe294107590 legacy=0x7fe2941068c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:50.793 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.790+0000 7fe28affd640 1 -- 172.21.15.43:0/2241308727 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2941a9bd0 con 0x7fe29410df30 2024-03-21T13:25:50.793 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.790+0000 7fe29bc84640 1 -- 172.21.15.43:0/2241308727 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2941a6740 con 0x7fe29410df30 2024-03-21T13:25:50.793 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.790+0000 7fe29bc84640 1 -- 172.21.15.43:0/2241308727 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2941a6c10 con 0x7fe29410df30 2024-03-21T13:25:50.793 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.790+0000 7fe28affd640 1 -- 172.21.15.43:0/2241308727 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7fe290003e20 con 0x7fe29410df30 2024-03-21T13:25:50.794 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.790+0000 7fe28affd640 1 -- 172.21.15.43:0/2241308727 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fe290006000 con 0x7fe29410df30 2024-03-21T13:25:50.794 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.791+0000 7fe29bc84640 1 -- 172.21.15.43:0/2241308727 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe278005080 con 0x7fe29410df30 2024-03-21T13:25:50.794 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.792+0000 7fe28affd640 1 -- 172.21.15.43:0/2241308727 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7fe290003990 con 0x7fe29410df30 2024-03-21T13:25:50.794 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.792+0000 7fe28affd640 1 -- 172.21.15.43:0/2241308727 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(813..813 src has 251..813) v4 ==== 7785+0+0 (unknown 4206456813 0 0) 0x7fe29008a200 con 0x7fe29410df30 2024-03-21T13:25:50.796 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.797+0000 7fe28affd640 1 -- 172.21.15.43:0/2241308727 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7fe2900571e0 con 0x7fe29410df30 2024-03-21T13:25:50.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:50 smithi155 ceph-mon[33846]: from='client.58700 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "10"}]': finished 2024-03-21T13:25:50.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:50 smithi155 ceph-mon[33846]: osdmap e812: 8 total, 8 up, 8 in 2024-03-21T13:25:50.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:50 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/115109259' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:25:50.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:50 smithi155 ceph-mon[33846]: from='client.58700 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:25:50.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:50 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:25:50.982 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:50.982+0000 7fe29bc84640 1 -- 172.21.15.43:0/2241308727 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "osd pool application enable", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "app": "rados"} v 0) v1 -- 0x7fe278005370 con 0x7fe29410df30 2024-03-21T13:25:50.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:50 smithi043 ceph-mon[30062]: from='client.58700 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "10"}]': finished 2024-03-21T13:25:50.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:50 smithi043 ceph-mon[30062]: osdmap e812: 8 total, 8 up, 8 in 2024-03-21T13:25:50.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:50 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/115109259' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:25:50.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:50 smithi043 ceph-mon[30062]: from='client.58700 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:25:50.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:50 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:25:50.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:50 smithi043 ceph-mon[41699]: from='client.58700 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "10"}]': finished 2024-03-21T13:25:50.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:50 smithi043 ceph-mon[41699]: osdmap e812: 8 total, 8 up, 8 in 2024-03-21T13:25:50.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:50 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/115109259' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:25:50.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:50 smithi043 ceph-mon[41699]: from='client.58700 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:25:50.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:50 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:25:51.680 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:51.681+0000 7fe28affd640 1 -- 172.21.15.43:0/2241308727 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "app": "rados"}]=0 enabled application 'rados' on pool 'dfa6c75f-2043-4860-bf82-1f19a1404959' v814) v1 ==== 213+0+0 (unknown 3446897524 0 0) 0x7fe29005ac30 con 0x7fe29410df30 2024-03-21T13:25:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:51 smithi155 ceph-mon[33846]: pgmap v1750: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:25:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:51 smithi155 ceph-mon[33846]: from='client.58700 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "10"}]': finished 2024-03-21T13:25:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:51 smithi155 ceph-mon[33846]: osdmap e813: 8 total, 8 up, 8 in 2024-03-21T13:25:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:51 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:25:51.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:51 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2241308727' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "app": "rados"}]: dispatch 2024-03-21T13:25:51.799 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:51.799+0000 7fe29bc84640 1 -- 172.21.15.43:0/2241308727 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "osd pool application enable", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "app": "rados"} v 0) v1 -- 0x7fe278001090 con 0x7fe29410df30 2024-03-21T13:25:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:51 smithi043 ceph-mon[30062]: pgmap v1750: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:25:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:51 smithi043 ceph-mon[30062]: from='client.58700 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "10"}]': finished 2024-03-21T13:25:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:51 smithi043 ceph-mon[30062]: osdmap e813: 8 total, 8 up, 8 in 2024-03-21T13:25:51.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:51 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:25:51.985 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:51 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2241308727' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "app": "rados"}]: dispatch 2024-03-21T13:25:51.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:51 smithi043 ceph-mon[41699]: pgmap v1750: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 887 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:25:51.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:51 smithi043 ceph-mon[41699]: from='client.58700 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "10"}]': finished 2024-03-21T13:25:51.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:51 smithi043 ceph-mon[41699]: osdmap e813: 8 total, 8 up, 8 in 2024-03-21T13:25:51.985 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:51 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:25:51.986 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:51 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2241308727' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "app": "rados"}]: dispatch 2024-03-21T13:25:52.692 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:52.692+0000 7fe28affd640 1 -- 172.21.15.43:0/2241308727 <== mon.0 v1:172.21.15.43:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "app": "rados"}]=0 enabled application 'rados' on pool 'dfa6c75f-2043-4860-bf82-1f19a1404959' v815) v1 ==== 213+0+0 (unknown 4149382546 0 0) 0x7fe29005f4f0 con 0x7fe29410df30 2024-03-21T13:25:52.693 INFO:tasks.workunit.client.0.smithi043.stderr:enabled application 'rados' on pool 'dfa6c75f-2043-4860-bf82-1f19a1404959' 2024-03-21T13:25:52.696 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:52.697+0000 7fe29bc84640 1 -- 172.21.15.43:0/2241308727 >> v1:172.21.15.43:6800/4199387216 conn(0x7fe28006f2d0 legacy=0x7fe280071790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:52.696 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:52.697+0000 7fe29bc84640 1 -- 172.21.15.43:0/2241308727 >> v1:172.21.15.43:6789/0 conn(0x7fe29410df30 legacy=0x7fe2941a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:25:52.697 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:52.697+0000 7fe29bc84640 1 -- 172.21.15.43:0/2241308727 shutdown_connections 2024-03-21T13:25:52.697 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:52.697+0000 7fe29bc84640 1 -- 172.21.15.43:0/2241308727 >> 172.21.15.43:0/2241308727 conn(0x7fe2940fcfa0 msgr2=0x7fe29410e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:25:52.697 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:52.697+0000 7fe29bc84640 1 -- 172.21.15.43:0/2241308727 shutdown_connections 2024-03-21T13:25:52.697 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:25:52.697+0000 7fe29bc84640 1 -- 172.21.15.43:0/2241308727 wait complete. 2024-03-21T13:25:52.710 INFO:tasks.workunit.client.0.smithi043.stderr:++ seq 1 10 2024-03-21T13:25:52.720 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-03-21T13:25:52.720 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p dfa6c75f-2043-4860-bf82-1f19a1404959 put obj1 /etc/passwd 2024-03-21T13:25:52.756 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-03-21T13:25:52.756 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p dfa6c75f-2043-4860-bf82-1f19a1404959 put obj2 /etc/passwd 2024-03-21T13:25:52.794 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-03-21T13:25:52.794 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p dfa6c75f-2043-4860-bf82-1f19a1404959 put obj3 /etc/passwd 2024-03-21T13:25:52.827 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-03-21T13:25:52.828 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p dfa6c75f-2043-4860-bf82-1f19a1404959 put obj4 /etc/passwd 2024-03-21T13:25:52.861 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-03-21T13:25:52.861 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p dfa6c75f-2043-4860-bf82-1f19a1404959 put obj5 /etc/passwd 2024-03-21T13:25:52.893 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-03-21T13:25:52.893 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p dfa6c75f-2043-4860-bf82-1f19a1404959 put obj6 /etc/passwd 2024-03-21T13:25:52.925 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-03-21T13:25:52.925 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p dfa6c75f-2043-4860-bf82-1f19a1404959 put obj7 /etc/passwd 2024-03-21T13:25:52.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2241308727' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "app": "rados"}]': finished 2024-03-21T13:25:52.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:52 smithi155 ceph-mon[33846]: osdmap e814: 8 total, 8 up, 8 in 2024-03-21T13:25:52.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:52 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2241308727' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "app": "rados"}]: dispatch 2024-03-21T13:25:52.958 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-03-21T13:25:52.958 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p dfa6c75f-2043-4860-bf82-1f19a1404959 put obj8 /etc/passwd 2024-03-21T13:25:52.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2241308727' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "app": "rados"}]': finished 2024-03-21T13:25:52.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:52 smithi043 ceph-mon[30062]: osdmap e814: 8 total, 8 up, 8 in 2024-03-21T13:25:52.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:52 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2241308727' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "app": "rados"}]: dispatch 2024-03-21T13:25:52.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2241308727' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "app": "rados"}]': finished 2024-03-21T13:25:52.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:52 smithi043 ceph-mon[41699]: osdmap e814: 8 total, 8 up, 8 in 2024-03-21T13:25:52.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:52 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2241308727' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "app": "rados"}]: dispatch 2024-03-21T13:25:52.991 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-03-21T13:25:52.991 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p dfa6c75f-2043-4860-bf82-1f19a1404959 put obj9 /etc/passwd 2024-03-21T13:25:53.023 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-03-21T13:25:53.023 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p dfa6c75f-2043-4860-bf82-1f19a1404959 put obj10 /etc/passwd 2024-03-21T13:25:53.056 INFO:tasks.workunit.client.0.smithi043.stderr:+ sleep 30 2024-03-21T13:25:53.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:53 smithi155 ceph-mon[33846]: pgmap v1753: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 887 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:25:53.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:53 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2241308727' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "app": "rados"}]': finished 2024-03-21T13:25:53.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:53 smithi155 ceph-mon[33846]: osdmap e815: 8 total, 8 up, 8 in 2024-03-21T13:25:53.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:53 smithi043 ceph-mon[30062]: pgmap v1753: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 887 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:25:53.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:53 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2241308727' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "app": "rados"}]': finished 2024-03-21T13:25:53.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:53 smithi043 ceph-mon[30062]: osdmap e815: 8 total, 8 up, 8 in 2024-03-21T13:25:53.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:53 smithi043 ceph-mon[41699]: pgmap v1753: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 887 MiB used, 714 GiB / 715 GiB avail 2024-03-21T13:25:53.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:53 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2241308727' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "app": "rados"}]': finished 2024-03-21T13:25:53.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:53 smithi043 ceph-mon[41699]: osdmap e815: 8 total, 8 up, 8 in 2024-03-21T13:25:55.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:55 smithi155 ceph-mon[33846]: pgmap v1755: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 24 KiB/s wr, 3 op/s 2024-03-21T13:25:55.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:55 smithi043 ceph-mon[30062]: pgmap v1755: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 24 KiB/s wr, 3 op/s 2024-03-21T13:25:55.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:55 smithi043 ceph-mon[41699]: pgmap v1755: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 24 KiB/s wr, 3 op/s 2024-03-21T13:25:57.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:57 smithi155 ceph-mon[33846]: pgmap v1756: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 20 KiB/s wr, 2 op/s 2024-03-21T13:25:57.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:57 smithi043 ceph-mon[30062]: pgmap v1756: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 20 KiB/s wr, 2 op/s 2024-03-21T13:25:57.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:57 smithi043 ceph-mon[41699]: pgmap v1756: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 20 KiB/s wr, 2 op/s 2024-03-21T13:25:58.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:25:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:25:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:25:58.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:58 smithi155 ceph-mon[33846]: pool 'dfa6c75f-2043-4860-bf82-1f19a1404959' is full (reached quota's max_objects: 10) 2024-03-21T13:25:58.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:58 smithi155 ceph-mon[33846]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-21T13:25:58.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:58 smithi155 ceph-mon[33846]: osdmap e816: 8 total, 8 up, 8 in 2024-03-21T13:25:58.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:58 smithi043 ceph-mon[30062]: pool 'dfa6c75f-2043-4860-bf82-1f19a1404959' is full (reached quota's max_objects: 10) 2024-03-21T13:25:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:58 smithi043 ceph-mon[30062]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-21T13:25:58.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:58 smithi043 ceph-mon[30062]: osdmap e816: 8 total, 8 up, 8 in 2024-03-21T13:25:58.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:58 smithi043 ceph-mon[41699]: pool 'dfa6c75f-2043-4860-bf82-1f19a1404959' is full (reached quota's max_objects: 10) 2024-03-21T13:25:58.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:58 smithi043 ceph-mon[41699]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-21T13:25:58.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:58 smithi043 ceph-mon[41699]: osdmap e816: 8 total, 8 up, 8 in 2024-03-21T13:25:59.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:25:59 smithi155 ceph-mon[33846]: pgmap v1758: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 17 KiB/s wr, 2 op/s 2024-03-21T13:25:59.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:25:59 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:25:59.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:25:59 smithi043 ceph-mon[30062]: pgmap v1758: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 17 KiB/s wr, 2 op/s 2024-03-21T13:25:59.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:25:59 smithi043 ceph-mon[41699]: pgmap v1758: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 17 KiB/s wr, 2 op/s 2024-03-21T13:26:00.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:00 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:00.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:00 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:00 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:01.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:01 smithi043 ceph-mon[30062]: pgmap v1759: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-03-21T13:26:01.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:01 smithi043 ceph-mon[41699]: pgmap v1759: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-03-21T13:26:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:01 smithi155 ceph-mon[33846]: pgmap v1759: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-03-21T13:26:03.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:03 smithi043 ceph-mon[41699]: pgmap v1760: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 12 KiB/s wr, 2 op/s 2024-03-21T13:26:03.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:03 smithi043 ceph-mon[30062]: pgmap v1760: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 12 KiB/s wr, 2 op/s 2024-03-21T13:26:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:03 smithi155 ceph-mon[33846]: pgmap v1760: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 12 KiB/s wr, 2 op/s 2024-03-21T13:26:05.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:05 smithi043 ceph-mon[30062]: pgmap v1761: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:26:05.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:05 smithi043 ceph-mon[41699]: pgmap v1761: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:26:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:05 smithi155 ceph-mon[33846]: pgmap v1761: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:26:07.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:06 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:26:07.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:06 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:26:07.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:06 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:26:07.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:06 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:26:07.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:06 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:26:07.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:06 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:26:08.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:07 smithi043 ceph-mon[30062]: pgmap v1762: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:26:08.024 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:07 smithi043 ceph-mon[41699]: pgmap v1762: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:26:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:07 smithi155 ceph-mon[33846]: pgmap v1762: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:26:08.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:26:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:26:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:26:10.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:09 smithi155 ceph-mon[33846]: pgmap v1763: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 997 B/s rd, 0 op/s 2024-03-21T13:26:10.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:26:09 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:26:10.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:09 smithi043 ceph-mon[30062]: pgmap v1763: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 997 B/s rd, 0 op/s 2024-03-21T13:26:10.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:09 smithi043 ceph-mon[41699]: pgmap v1763: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 997 B/s rd, 0 op/s 2024-03-21T13:26:11.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:10 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:11.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:10 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:11.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:10 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:11 smithi155 ceph-mon[33846]: pgmap v1764: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:12.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:11 smithi043 ceph-mon[30062]: pgmap v1764: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:12.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:11 smithi043 ceph-mon[41699]: pgmap v1764: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:13 smithi155 ceph-mon[33846]: pgmap v1765: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:14.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:13 smithi043 ceph-mon[30062]: pgmap v1765: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:14.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:13 smithi043 ceph-mon[41699]: pgmap v1765: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:16.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:15 smithi155 ceph-mon[33846]: pgmap v1766: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:26:16.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:15 smithi043 ceph-mon[30062]: pgmap v1766: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:26:16.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:15 smithi043 ceph-mon[41699]: pgmap v1766: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:26:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:17 smithi155 ceph-mon[33846]: pgmap v1767: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:18.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:17 smithi043 ceph-mon[30062]: pgmap v1767: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:18.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:26:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:26:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:26:18.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:17 smithi043 ceph-mon[41699]: pgmap v1767: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:19 smithi155 ceph-mon[33846]: pgmap v1768: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:26:20.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:26:19 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:26:20.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:19 smithi043 ceph-mon[30062]: pgmap v1768: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:26:20.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:19 smithi043 ceph-mon[41699]: pgmap v1768: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:26:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:20 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:20 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:26:21.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:20 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:21.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:20 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:26:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:20 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:20 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:26:22.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:21 smithi155 ceph-mon[33846]: pgmap v1769: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:22.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:21 smithi043 ceph-mon[30062]: pgmap v1769: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:22.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:21 smithi043 ceph-mon[41699]: pgmap v1769: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:23.058 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=158892 2024-03-21T13:26:23.058 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota dfa6c75f-2043-4860-bf82-1f19a1404959 max_objects 100 2024-03-21T13:26:23.058 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p dfa6c75f-2043-4860-bf82-1f19a1404959 put onemore /etc/passwd 2024-03-21T13:26:23.156 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.153+0000 7f196c5c7640 1 -- 172.21.15.43:0/1233830566 >> v1:172.21.15.43:6789/0 conn(0x7f196410df50 legacy=0x7f1964110410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:23.156 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.153+0000 7f196c5c7640 1 -- 172.21.15.43:0/1233830566 shutdown_connections 2024-03-21T13:26:23.156 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.153+0000 7f196c5c7640 1 -- 172.21.15.43:0/1233830566 >> 172.21.15.43:0/1233830566 conn(0x7f19640fcf80 msgr2=0x7f19640ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:23.156 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.153+0000 7f196c5c7640 1 -- 172.21.15.43:0/1233830566 shutdown_connections 2024-03-21T13:26:23.156 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.153+0000 7f196c5c7640 1 -- 172.21.15.43:0/1233830566 wait complete. 2024-03-21T13:26:23.157 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.153+0000 7f196c5c7640 1 Processor -- start 2024-03-21T13:26:23.157 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.153+0000 7f196c5c7640 1 -- start start 2024-03-21T13:26:23.157 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.153+0000 7f196c5c7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f19641a65f0 con 0x7f196410df50 2024-03-21T13:26:23.157 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.154+0000 7f196c5c7640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f19641a7830 con 0x7f19641075b0 2024-03-21T13:26:23.157 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.154+0000 7f196c5c7640 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f19641a8a70 con 0x7f196410a350 2024-03-21T13:26:23.157 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.154+0000 7f1969b62640 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7f196410a350 0x7f19641a13c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:37320/0 (socket says 172.21.15.43:37320) 2024-03-21T13:26:23.157 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.154+0000 7f1969b62640 1 -- 172.21.15.43:0/3818302601 learned_addr learned my addr 172.21.15.43:0/3818302601 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T13:26:23.157 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.154+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 121353123 0 0) 0x7f19641a65f0 con 0x7f196410df50 2024-03-21T13:26:23.157 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.154+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f194c003160 con 0x7f196410df50 2024-03-21T13:26:23.157 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.154+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3726692264 0 0) 0x7f19641a8a70 con 0x7f196410a350 2024-03-21T13:26:23.157 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.154+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19641a65f0 con 0x7f196410a350 2024-03-21T13:26:23.157 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.154+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 288652242 0 0) 0x7f19641a7830 con 0x7f19641075b0 2024-03-21T13:26:23.157 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.154+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19641a8a70 con 0x7f19641075b0 2024-03-21T13:26:23.157 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.154+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2499470953 0 0) 0x7f194c003160 con 0x7f196410df50 2024-03-21T13:26:23.158 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.154+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f19641a7830 con 0x7f196410df50 2024-03-21T13:26:23.158 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.154+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f1954001b80 con 0x7f196410df50 2024-03-21T13:26:23.158 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.154+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3117524889 0 0) 0x7f19641a65f0 con 0x7f196410a350 2024-03-21T13:26:23.158 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.154+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f194c003160 con 0x7f196410a350 2024-03-21T13:26:23.158 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.154+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3064733509 0 0) 0x7f19641a8a70 con 0x7f19641075b0 2024-03-21T13:26:23.158 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.154+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f19641a65f0 con 0x7f19641075b0 2024-03-21T13:26:23.158 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.154+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f1958001870 con 0x7f196410a350 2024-03-21T13:26:23.158 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.154+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f195c001890 con 0x7f19641075b0 2024-03-21T13:26:23.158 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.154+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2385630855 0 0) 0x7f19641a7830 con 0x7f196410df50 2024-03-21T13:26:23.158 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.154+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 >> v1:172.21.15.43:6790/0 conn(0x7f196410a350 legacy=0x7f19641a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:23.158 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.154+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 >> v1:172.21.15.155:6789/0 conn(0x7f19641075b0 legacy=0x7f19641069a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:23.158 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.154+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19641a9cb0 con 0x7f196410df50 2024-03-21T13:26:23.158 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.154+0000 7f196c5c7640 1 -- 172.21.15.43:0/3818302601 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19641a8ca0 con 0x7f196410df50 2024-03-21T13:26:23.158 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.155+0000 7f196c5c7640 1 -- 172.21.15.43:0/3818302601 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f19641a9170 con 0x7f196410df50 2024-03-21T13:26:23.158 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.155+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f1954004910 con 0x7f196410df50 2024-03-21T13:26:23.159 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.155+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f1954004e30 con 0x7f196410df50 2024-03-21T13:26:23.159 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.156+0000 7f196c5c7640 1 -- 172.21.15.43:0/3818302601 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1950005080 con 0x7f196410df50 2024-03-21T13:26:23.159 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.156+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7f1954001e20 con 0x7f196410df50 2024-03-21T13:26:23.159 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.157+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(816..816 src has 251..816) v4 ==== 7798+0+0 (unknown 677048526 0 0) 0x7f1954088d30 con 0x7f196410df50 2024-03-21T13:26:23.159 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.157+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=817}) v3 -- 0x7f19641a7830 con 0x7f196410df50 2024-03-21T13:26:23.162 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.161+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f1954055d40 con 0x7f196410df50 2024-03-21T13:26:23.346 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.345+0000 7f196c5c7640 1 -- 172.21.15.43:0/3818302601 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f1950005370 con 0x7f196410df50 2024-03-21T13:26:23.798 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.797+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool dfa6c75f-2043-4860-bf82-1f19a1404959 v817) v1 ==== 225+0+0 (unknown 1006887098 0 0) 0x7f1954059790 con 0x7f196410df50 2024-03-21T13:26:23.798 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.797+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 <== mon.0 v1:172.21.15.43:6789/0 11 ==== osd_map(817..817 src has 251..817) v4 ==== 628+0+0 (unknown 3713880506 0 0) 0x7f1954086b20 con 0x7f196410df50 2024-03-21T13:26:23.798 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.797+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=818}) v3 -- 0x7f19641a8a70 con 0x7f196410df50 2024-03-21T13:26:23.915 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:23.914+0000 7f196c5c7640 1 -- 172.21.15.43:0/3818302601 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f1950005370 con 0x7f196410df50 2024-03-21T13:26:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:23 smithi155 ceph-mon[33846]: pgmap v1770: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:23 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3818302601' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "100"}]: dispatch 2024-03-21T13:26:24.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:23 smithi043 ceph-mon[30062]: pgmap v1770: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:24.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:23 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3818302601' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "100"}]: dispatch 2024-03-21T13:26:24.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:23 smithi043 ceph-mon[41699]: pgmap v1770: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:24.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:23 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3818302601' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "100"}]: dispatch 2024-03-21T13:26:24.812 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:24.811+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 <== mon.0 v1:172.21.15.43:6789/0 12 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool dfa6c75f-2043-4860-bf82-1f19a1404959 v818) v1 ==== 225+0+0 (unknown 1160160306 0 0) 0x7f195405e050 con 0x7f196410df50 2024-03-21T13:26:24.812 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_objects = 100 for pool dfa6c75f-2043-4860-bf82-1f19a1404959 2024-03-21T13:26:24.812 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:24.811+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 <== mon.0 v1:172.21.15.43:6789/0 13 ==== osd_map(818..818 src has 251..818) v4 ==== 628+0+0 (unknown 245333753 0 0) 0x7f1954051680 con 0x7f196410df50 2024-03-21T13:26:24.812 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:24.811+0000 7f19637fe640 1 -- 172.21.15.43:0/3818302601 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=819}) v3 -- 0x7f194c07c4d0 con 0x7f196410df50 2024-03-21T13:26:24.816 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:24.815+0000 7f196c5c7640 1 -- 172.21.15.43:0/3818302601 >> v1:172.21.15.43:6800/4199387216 conn(0x7f194c06f260 legacy=0x7f194c071720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:24.817 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:24.816+0000 7f196c5c7640 1 -- 172.21.15.43:0/3818302601 >> v1:172.21.15.43:6789/0 conn(0x7f196410df50 legacy=0x7f19641a4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:24.817 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:24.816+0000 7f196c5c7640 1 -- 172.21.15.43:0/3818302601 shutdown_connections 2024-03-21T13:26:24.817 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:24.816+0000 7f196c5c7640 1 -- 172.21.15.43:0/3818302601 >> 172.21.15.43:0/3818302601 conn(0x7f19640fcf80 msgr2=0x7f1964111120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:24.817 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:24.816+0000 7f196c5c7640 1 -- 172.21.15.43:0/3818302601 shutdown_connections 2024-03-21T13:26:24.817 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:24.816+0000 7f196c5c7640 1 -- 172.21.15.43:0/3818302601 wait complete. 2024-03-21T13:26:24.830 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 158892 2024-03-21T13:26:25.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:24 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3818302601' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "100"}]': finished 2024-03-21T13:26:25.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:24 smithi155 ceph-mon[33846]: osdmap e817: 8 total, 8 up, 8 in 2024-03-21T13:26:25.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:24 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3818302601' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "100"}]: dispatch 2024-03-21T13:26:25.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:24 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3818302601' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "100"}]': finished 2024-03-21T13:26:25.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:24 smithi043 ceph-mon[30062]: osdmap e817: 8 total, 8 up, 8 in 2024-03-21T13:26:25.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:24 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3818302601' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "100"}]: dispatch 2024-03-21T13:26:25.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:24 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3818302601' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "100"}]': finished 2024-03-21T13:26:25.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:24 smithi043 ceph-mon[41699]: osdmap e817: 8 total, 8 up, 8 in 2024-03-21T13:26:25.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:24 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3818302601' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "100"}]: dispatch 2024-03-21T13:26:26.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:25 smithi155 ceph-mon[33846]: pgmap v1772: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:26:26.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:25 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3818302601' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "100"}]': finished 2024-03-21T13:26:26.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:25 smithi155 ceph-mon[33846]: osdmap e818: 8 total, 8 up, 8 in 2024-03-21T13:26:26.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:25 smithi043 ceph-mon[30062]: pgmap v1772: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:26:26.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:25 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3818302601' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "100"}]': finished 2024-03-21T13:26:26.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:25 smithi043 ceph-mon[30062]: osdmap e818: 8 total, 8 up, 8 in 2024-03-21T13:26:26.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:25 smithi043 ceph-mon[41699]: pgmap v1772: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:26:26.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:25 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3818302601' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "100"}]': finished 2024-03-21T13:26:26.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:25 smithi043 ceph-mon[41699]: osdmap e818: 8 total, 8 up, 8 in 2024-03-21T13:26:28.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:27 smithi155 ceph-mon[33846]: pgmap v1774: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:26:28.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:27 smithi043 ceph-mon[30062]: pgmap v1774: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:26:28.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:26:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:26:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:26:28.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:27 smithi043 ceph-mon[41699]: pgmap v1774: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:26:28.356 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 0 -ne 0 ']' 2024-03-21T13:26:28.357 INFO:tasks.workunit.client.0.smithi043.stderr:+ true 2024-03-21T13:26:28.357 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p dfa6c75f-2043-4860-bf82-1f19a1404959 put twomore /etc/passwd 2024-03-21T13:26:28.397 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota dfa6c75f-2043-4860-bf82-1f19a1404959 max_bytes 100 2024-03-21T13:26:28.485 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.481+0000 7f515234b640 1 -- 172.21.15.43:0/3569190952 >> v1:172.21.15.43:6790/0 conn(0x7f514c1075c0 legacy=0x7f514c1079a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:28.485 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.481+0000 7f515234b640 1 -- 172.21.15.43:0/3569190952 shutdown_connections 2024-03-21T13:26:28.485 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.481+0000 7f515234b640 1 -- 172.21.15.43:0/3569190952 >> 172.21.15.43:0/3569190952 conn(0x7f514c0fcff0 msgr2=0x7f514c0ff410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:28.485 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.481+0000 7f515234b640 1 -- 172.21.15.43:0/3569190952 shutdown_connections 2024-03-21T13:26:28.485 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.481+0000 7f515234b640 1 -- 172.21.15.43:0/3569190952 wait complete. 2024-03-21T13:26:28.485 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.481+0000 7f515234b640 1 Processor -- start 2024-03-21T13:26:28.485 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.481+0000 7f515234b640 1 -- start start 2024-03-21T13:26:28.485 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.482+0000 7f515234b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f514c1a87d0 con 0x7f514c10df60 2024-03-21T13:26:28.485 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.482+0000 7f515234b640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f514c1a99f0 con 0x7f514c10a360 2024-03-21T13:26:28.485 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.482+0000 7f515234b640 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f514c1aac30 con 0x7f514c1075c0 2024-03-21T13:26:28.485 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.482+0000 7f514bfff640 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7f514c1075c0 0x7f514c1068f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:37368/0 (socket says 172.21.15.43:37368) 2024-03-21T13:26:28.486 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.482+0000 7f514bfff640 1 -- 172.21.15.43:0/2136190907 learned_addr learned my addr 172.21.15.43:0/2136190907 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T13:26:28.486 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.482+0000 7f51497fa640 1 -- 172.21.15.43:0/2136190907 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1425539909 0 0) 0x7f514c1aac30 con 0x7f514c1075c0 2024-03-21T13:26:28.486 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.482+0000 7f51497fa640 1 -- 172.21.15.43:0/2136190907 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5134003160 con 0x7f514c1075c0 2024-03-21T13:26:28.486 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.482+0000 7f51497fa640 1 -- 172.21.15.43:0/2136190907 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 250688954 0 0) 0x7f514c1a87d0 con 0x7f514c10df60 2024-03-21T13:26:28.486 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.482+0000 7f51497fa640 1 -- 172.21.15.43:0/2136190907 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f514c1aac30 con 0x7f514c10df60 2024-03-21T13:26:28.486 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.482+0000 7f51497fa640 1 -- 172.21.15.43:0/2136190907 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2553613315 0 0) 0x7f5134003160 con 0x7f514c1075c0 2024-03-21T13:26:28.486 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.482+0000 7f51497fa640 1 -- 172.21.15.43:0/2136190907 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f514c1a87d0 con 0x7f514c1075c0 2024-03-21T13:26:28.486 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.482+0000 7f51497fa640 1 -- 172.21.15.43:0/2136190907 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f513c0043e0 con 0x7f514c1075c0 2024-03-21T13:26:28.486 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.482+0000 7f51497fa640 1 -- 172.21.15.43:0/2136190907 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1632492271 0 0) 0x7f514c1a99f0 con 0x7f514c10a360 2024-03-21T13:26:28.486 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.482+0000 7f51497fa640 1 -- 172.21.15.43:0/2136190907 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5134003160 con 0x7f514c10a360 2024-03-21T13:26:28.486 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.482+0000 7f51497fa640 1 -- 172.21.15.43:0/2136190907 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2769970740 0 0) 0x7f514c1aac30 con 0x7f514c10df60 2024-03-21T13:26:28.486 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.482+0000 7f51497fa640 1 -- 172.21.15.43:0/2136190907 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f514c1a99f0 con 0x7f514c10df60 2024-03-21T13:26:28.486 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.482+0000 7f51497fa640 1 -- 172.21.15.43:0/2136190907 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f5144001bb0 con 0x7f514c10df60 2024-03-21T13:26:28.486 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.482+0000 7f51497fa640 1 -- 172.21.15.43:0/2136190907 <== mon.2 v1:172.21.15.43:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 414861614 0 0) 0x7f514c1a87d0 con 0x7f514c1075c0 2024-03-21T13:26:28.487 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.482+0000 7f51497fa640 1 -- 172.21.15.43:0/2136190907 >> v1:172.21.15.155:6789/0 conn(0x7f514c10a360 legacy=0x7f514c1a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:28.487 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.482+0000 7f51497fa640 1 -- 172.21.15.43:0/2136190907 >> v1:172.21.15.43:6789/0 conn(0x7f514c10df60 legacy=0x7f514c1a6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:28.487 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.482+0000 7f51497fa640 1 -- 172.21.15.43:0/2136190907 --> v1:172.21.15.43:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514c1abe70 con 0x7f514c1075c0 2024-03-21T13:26:28.487 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.483+0000 7f515234b640 1 -- 172.21.15.43:0/2136190907 --> v1:172.21.15.43:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514c1a8a00 con 0x7f514c1075c0 2024-03-21T13:26:28.487 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.483+0000 7f515234b640 1 -- 172.21.15.43:0/2136190907 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f514c1a8ed0 con 0x7f514c1075c0 2024-03-21T13:26:28.487 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.483+0000 7f51497fa640 1 -- 172.21.15.43:0/2136190907 <== mon.2 v1:172.21.15.43:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f513c001600 con 0x7f514c1075c0 2024-03-21T13:26:28.487 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.483+0000 7f51497fa640 1 -- 172.21.15.43:0/2136190907 <== mon.2 v1:172.21.15.43:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f513c004ed0 con 0x7f514c1075c0 2024-03-21T13:26:28.487 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.484+0000 7f515234b640 1 -- 172.21.15.43:0/2136190907 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5138005080 con 0x7f514c1075c0 2024-03-21T13:26:28.487 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.484+0000 7f51497fa640 1 -- 172.21.15.43:0/2136190907 <== mon.2 v1:172.21.15.43:6790/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7f513c0037d0 con 0x7f514c1075c0 2024-03-21T13:26:28.487 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.484+0000 7f51497fa640 1 -- 172.21.15.43:0/2136190907 <== mon.2 v1:172.21.15.43:6790/0 8 ==== osd_map(819..819 src has 251..819) v4 ==== 7798+0+0 (unknown 360702972 0 0) 0x7f513c089f70 con 0x7f514c1075c0 2024-03-21T13:26:28.489 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.489+0000 7f51497fa640 1 -- 172.21.15.43:0/2136190907 <== mon.2 v1:172.21.15.43:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f513c056f40 con 0x7f514c1075c0 2024-03-21T13:26:28.679 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:28.678+0000 7f515234b640 1 -- 172.21.15.43:0/2136190907 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f5138005370 con 0x7f514c1075c0 2024-03-21T13:26:29.213 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:28 smithi155 ceph-mon[33846]: pool 'dfa6c75f-2043-4860-bf82-1f19a1404959' no longer out of quota; removing NO_QUOTA flag 2024-03-21T13:26:29.213 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:28 smithi155 ceph-mon[33846]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-21T13:26:29.213 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:28 smithi155 ceph-mon[33846]: osdmap e819: 8 total, 8 up, 8 in 2024-03-21T13:26:29.213 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:28 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2136190907' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-21T13:26:29.213 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:28 smithi155 ceph-mon[33846]: from='client.58826 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-21T13:26:29.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:28 smithi043 ceph-mon[30062]: pool 'dfa6c75f-2043-4860-bf82-1f19a1404959' no longer out of quota; removing NO_QUOTA flag 2024-03-21T13:26:29.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:28 smithi043 ceph-mon[30062]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-21T13:26:29.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:28 smithi043 ceph-mon[30062]: osdmap e819: 8 total, 8 up, 8 in 2024-03-21T13:26:29.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:28 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2136190907' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-21T13:26:29.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:28 smithi043 ceph-mon[30062]: from='client.58826 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-21T13:26:29.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:28 smithi043 ceph-mon[41699]: pool 'dfa6c75f-2043-4860-bf82-1f19a1404959' no longer out of quota; removing NO_QUOTA flag 2024-03-21T13:26:29.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:28 smithi043 ceph-mon[41699]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-21T13:26:29.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:28 smithi043 ceph-mon[41699]: osdmap e819: 8 total, 8 up, 8 in 2024-03-21T13:26:29.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:28 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2136190907' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-21T13:26:29.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:28 smithi043 ceph-mon[41699]: from='client.58826 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-21T13:26:29.361 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:29.360+0000 7f51497fa640 1 -- 172.21.15.43:0/2136190907 <== mon.2 v1:172.21.15.43:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool dfa6c75f-2043-4860-bf82-1f19a1404959 v820) v1 ==== 221+0+0 (unknown 971658177 0 0) 0x7f513c05a990 con 0x7f514c1075c0 2024-03-21T13:26:29.479 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:29.479+0000 7f515234b640 1 -- 172.21.15.43:0/2136190907 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f5138000c40 con 0x7f514c1075c0 2024-03-21T13:26:30.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:26:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:26:30.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:29 smithi155 ceph-mon[33846]: pgmap v1776: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:26:30.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:29 smithi155 ceph-mon[33846]: from='client.58826 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "100"}]': finished 2024-03-21T13:26:30.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:29 smithi155 ceph-mon[33846]: osdmap e820: 8 total, 8 up, 8 in 2024-03-21T13:26:30.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:29 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2136190907' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-21T13:26:30.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:29 smithi155 ceph-mon[33846]: from='client.58826 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-21T13:26:30.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:29 smithi043 ceph-mon[30062]: pgmap v1776: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:26:30.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:29 smithi043 ceph-mon[30062]: from='client.58826 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "100"}]': finished 2024-03-21T13:26:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:29 smithi043 ceph-mon[30062]: osdmap e820: 8 total, 8 up, 8 in 2024-03-21T13:26:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:29 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2136190907' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-21T13:26:30.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:29 smithi043 ceph-mon[30062]: from='client.58826 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-21T13:26:30.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:29 smithi043 ceph-mon[41699]: pgmap v1776: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-21T13:26:30.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:29 smithi043 ceph-mon[41699]: from='client.58826 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "100"}]': finished 2024-03-21T13:26:30.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:29 smithi043 ceph-mon[41699]: osdmap e820: 8 total, 8 up, 8 in 2024-03-21T13:26:30.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:29 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2136190907' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-21T13:26:30.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:29 smithi043 ceph-mon[41699]: from='client.58826 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-21T13:26:30.363 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:30.362+0000 7f51497fa640 1 -- 172.21.15.43:0/2136190907 <== mon.2 v1:172.21.15.43:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool dfa6c75f-2043-4860-bf82-1f19a1404959 v821) v1 ==== 221+0+0 (unknown 3116007968 0 0) 0x7f513c05f250 con 0x7f514c1075c0 2024-03-21T13:26:30.363 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_bytes = 100 for pool dfa6c75f-2043-4860-bf82-1f19a1404959 2024-03-21T13:26:30.366 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:30.366+0000 7f515234b640 1 -- 172.21.15.43:0/2136190907 >> v1:172.21.15.43:6800/4199387216 conn(0x7f513406f080 legacy=0x7f5134071540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:30.367 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:30.366+0000 7f515234b640 1 -- 172.21.15.43:0/2136190907 >> v1:172.21.15.43:6790/0 conn(0x7f514c1075c0 legacy=0x7f514c1068f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:26:30.367 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:30.366+0000 7f515234b640 1 -- 172.21.15.43:0/2136190907 shutdown_connections 2024-03-21T13:26:30.367 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:30.366+0000 7f515234b640 1 -- 172.21.15.43:0/2136190907 >> 172.21.15.43:0/2136190907 conn(0x7f514c0fcff0 msgr2=0x7f514c10e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:26:30.367 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:30.366+0000 7f515234b640 1 -- 172.21.15.43:0/2136190907 shutdown_connections 2024-03-21T13:26:30.367 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:26:30.366+0000 7f515234b640 1 -- 172.21.15.43:0/2136190907 wait complete. 2024-03-21T13:26:30.378 INFO:tasks.workunit.client.0.smithi043.stderr:+ sleep 30 2024-03-21T13:26:31.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:30 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:31.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:30 smithi155 ceph-mon[33846]: from='client.58826 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "100"}]': finished 2024-03-21T13:26:31.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:30 smithi155 ceph-mon[33846]: osdmap e821: 8 total, 8 up, 8 in 2024-03-21T13:26:31.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:30 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:31.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:30 smithi043 ceph-mon[30062]: from='client.58826 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "100"}]': finished 2024-03-21T13:26:31.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:30 smithi043 ceph-mon[30062]: osdmap e821: 8 total, 8 up, 8 in 2024-03-21T13:26:31.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:30 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:31.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:30 smithi043 ceph-mon[41699]: from='client.58826 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "100"}]': finished 2024-03-21T13:26:31.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:30 smithi043 ceph-mon[41699]: osdmap e821: 8 total, 8 up, 8 in 2024-03-21T13:26:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:31 smithi155 ceph-mon[33846]: pgmap v1779: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 883 B/s rd, 0 op/s 2024-03-21T13:26:32.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:31 smithi043 ceph-mon[30062]: pgmap v1779: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 883 B/s rd, 0 op/s 2024-03-21T13:26:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:31 smithi043 ceph-mon[41699]: pgmap v1779: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 883 B/s rd, 0 op/s 2024-03-21T13:26:34.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:33 smithi155 ceph-mon[33846]: pgmap v1780: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:34.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:33 smithi155 ceph-mon[33846]: pool 'dfa6c75f-2043-4860-bf82-1f19a1404959' is full (reached quota's max_bytes: 100 B) 2024-03-21T13:26:34.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:33 smithi155 ceph-mon[33846]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-21T13:26:34.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:33 smithi155 ceph-mon[33846]: osdmap e822: 8 total, 8 up, 8 in 2024-03-21T13:26:34.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:33 smithi043 ceph-mon[30062]: pgmap v1780: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:34.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:33 smithi043 ceph-mon[30062]: pool 'dfa6c75f-2043-4860-bf82-1f19a1404959' is full (reached quota's max_bytes: 100 B) 2024-03-21T13:26:34.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:33 smithi043 ceph-mon[30062]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-21T13:26:34.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:33 smithi043 ceph-mon[30062]: osdmap e822: 8 total, 8 up, 8 in 2024-03-21T13:26:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:33 smithi043 ceph-mon[41699]: pgmap v1780: 176 pgs: 176 active+clean; 698 KiB data, 872 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:33 smithi043 ceph-mon[41699]: pool 'dfa6c75f-2043-4860-bf82-1f19a1404959' is full (reached quota's max_bytes: 100 B) 2024-03-21T13:26:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:33 smithi043 ceph-mon[41699]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-21T13:26:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:33 smithi043 ceph-mon[41699]: osdmap e822: 8 total, 8 up, 8 in 2024-03-21T13:26:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:35 smithi155 ceph-mon[33846]: pgmap v1782: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-03-21T13:26:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:26:36.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:35 smithi043 ceph-mon[30062]: pgmap v1782: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-03-21T13:26:36.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:26:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:35 smithi043 ceph-mon[41699]: pgmap v1782: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-03-21T13:26:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:26:38.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:37 smithi155 ceph-mon[33846]: pgmap v1783: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 705 B/s rd, 3.3 KiB/s wr, 0 op/s 2024-03-21T13:26:38.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:37 smithi043 ceph-mon[30062]: pgmap v1783: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 705 B/s rd, 3.3 KiB/s wr, 0 op/s 2024-03-21T13:26:38.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:26:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:26:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:26:38.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:37 smithi043 ceph-mon[41699]: pgmap v1783: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 705 B/s rd, 3.3 KiB/s wr, 0 op/s 2024-03-21T13:26:40.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:39 smithi155 ceph-mon[33846]: pgmap v1784: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-03-21T13:26:40.208 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:26:39 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:26:40.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:39 smithi043 ceph-mon[30062]: pgmap v1784: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-03-21T13:26:40.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:39 smithi043 ceph-mon[41699]: pgmap v1784: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-03-21T13:26:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:40 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:40 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:26:41.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:40 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:41.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:40 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:26:41.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:40 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:41.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:40 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:26:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:41 smithi155 ceph-mon[33846]: pgmap v1785: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-03-21T13:26:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:41 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:26:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:41 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:26:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:41 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:26:42.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:41 smithi043 ceph-mon[30062]: pgmap v1785: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-03-21T13:26:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:41 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:26:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:41 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:26:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:41 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:26:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:41 smithi043 ceph-mon[41699]: pgmap v1785: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-03-21T13:26:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:41 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:26:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:41 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:26:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:41 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:26:44.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:43 smithi155 ceph-mon[33846]: pgmap v1786: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-03-21T13:26:44.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:43 smithi043 ceph-mon[30062]: pgmap v1786: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-03-21T13:26:44.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:43 smithi043 ceph-mon[41699]: pgmap v1786: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-03-21T13:26:46.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:45 smithi155 ceph-mon[33846]: pgmap v1787: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 908 B/s rd, 0 op/s 2024-03-21T13:26:46.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:45 smithi043 ceph-mon[30062]: pgmap v1787: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 908 B/s rd, 0 op/s 2024-03-21T13:26:46.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:45 smithi043 ceph-mon[41699]: pgmap v1787: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 908 B/s rd, 0 op/s 2024-03-21T13:26:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:47 smithi155 ceph-mon[33846]: pgmap v1788: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:48.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:47 smithi043 ceph-mon[30062]: pgmap v1788: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:48.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:26:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:26:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:26:48.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:47 smithi043 ceph-mon[41699]: pgmap v1788: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:50.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:49 smithi155 ceph-mon[33846]: pgmap v1789: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:26:50.208 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:26:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:26:50.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:49 smithi043 ceph-mon[30062]: pgmap v1789: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:26:50.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:49 smithi043 ceph-mon[41699]: pgmap v1789: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:26:51.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:50 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:51.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:50 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:26:51.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:50 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:51.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:50 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:26:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:50 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:26:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:50 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:26:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:51 smithi155 ceph-mon[33846]: pgmap v1790: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:52.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:51 smithi043 ceph-mon[30062]: pgmap v1790: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:52.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:51 smithi043 ceph-mon[41699]: pgmap v1790: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:53 smithi155 ceph-mon[33846]: pgmap v1791: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:54.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:53 smithi043 ceph-mon[30062]: pgmap v1791: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:54.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:53 smithi043 ceph-mon[41699]: pgmap v1791: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:55 smithi155 ceph-mon[33846]: pgmap v1792: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:26:56.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:55 smithi043 ceph-mon[30062]: pgmap v1792: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:26:56.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:55 smithi043 ceph-mon[41699]: pgmap v1792: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:26:58.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:57 smithi155 ceph-mon[33846]: pgmap v1793: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:58.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:57 smithi043 ceph-mon[30062]: pgmap v1793: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:26:58.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:26:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:26:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:26:58.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:57 smithi043 ceph-mon[41699]: pgmap v1793: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:26:59 smithi155 ceph-mon[33846]: pgmap v1794: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:00.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:26:59 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:27:00.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:26:59 smithi043 ceph-mon[30062]: pgmap v1794: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:00.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:26:59 smithi043 ceph-mon[41699]: pgmap v1794: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:00.380 INFO:tasks.workunit.client.0.smithi043.stderr:+ pid=159111 2024-03-21T13:27:00.380 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota dfa6c75f-2043-4860-bf82-1f19a1404959 max_bytes 0 2024-03-21T13:27:00.380 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p dfa6c75f-2043-4860-bf82-1f19a1404959 put two /etc/passwd 2024-03-21T13:27:00.475 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.473+0000 7f69b375c640 1 -- 172.21.15.43:0/3773905595 >> v1:172.21.15.43:6790/0 conn(0x7f69ac0745c0 legacy=0x7f69ac10e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:00.476 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.473+0000 7f69b375c640 1 -- 172.21.15.43:0/3773905595 shutdown_connections 2024-03-21T13:27:00.476 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.473+0000 7f69b375c640 1 -- 172.21.15.43:0/3773905595 >> 172.21.15.43:0/3773905595 conn(0x7f69ac0fcf80 msgr2=0x7f69ac0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:00.476 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.473+0000 7f69b375c640 1 -- 172.21.15.43:0/3773905595 shutdown_connections 2024-03-21T13:27:00.476 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.473+0000 7f69b375c640 1 -- 172.21.15.43:0/3773905595 wait complete. 2024-03-21T13:27:00.476 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.473+0000 7f69b375c640 1 Processor -- start 2024-03-21T13:27:00.476 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.473+0000 7f69b375c640 1 -- start start 2024-03-21T13:27:00.476 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.473+0000 7f69b375c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f69ac1a87a0 con 0x7f69ac110170 2024-03-21T13:27:00.476 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.474+0000 7f69b375c640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f69ac1a99e0 con 0x7f69ac0745c0 2024-03-21T13:27:00.476 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.474+0000 7f69b375c640 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f69ac1aac20 con 0x7f69ac073ad0 2024-03-21T13:27:00.476 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.474+0000 7f69b14f8640 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7f69ac073ad0 0x7f69ac10e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:48426/0 (socket says 172.21.15.43:48426) 2024-03-21T13:27:00.476 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.474+0000 7f69b14f8640 1 -- 172.21.15.43:0/2048545697 learned_addr learned my addr 172.21.15.43:0/2048545697 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T13:27:00.477 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.474+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4107423142 0 0) 0x7f69ac1aac20 con 0x7f69ac073ad0 2024-03-21T13:27:00.477 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.474+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6998003520 con 0x7f69ac073ad0 2024-03-21T13:27:00.477 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.474+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 845877051 0 0) 0x7f69ac1a87a0 con 0x7f69ac110170 2024-03-21T13:27:00.477 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.474+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69ac1aac20 con 0x7f69ac110170 2024-03-21T13:27:00.477 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.474+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570389231 0 0) 0x7f69ac1a99e0 con 0x7f69ac0745c0 2024-03-21T13:27:00.477 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.474+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69ac1a87a0 con 0x7f69ac0745c0 2024-03-21T13:27:00.477 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.474+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2303757105 0 0) 0x7f6998003520 con 0x7f69ac073ad0 2024-03-21T13:27:00.477 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.474+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f69ac1a99e0 con 0x7f69ac073ad0 2024-03-21T13:27:00.477 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.474+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f69a80012d0 con 0x7f69ac073ad0 2024-03-21T13:27:00.477 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.474+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2356051362 0 0) 0x7f69ac1aac20 con 0x7f69ac110170 2024-03-21T13:27:00.477 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.474+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6998003520 con 0x7f69ac110170 2024-03-21T13:27:00.477 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.474+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f69a4001bb0 con 0x7f69ac110170 2024-03-21T13:27:00.477 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.474+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1285411672 0 0) 0x7f69ac1a87a0 con 0x7f69ac0745c0 2024-03-21T13:27:00.477 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.474+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f69ac1aac20 con 0x7f69ac0745c0 2024-03-21T13:27:00.477 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.474+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f699c001990 con 0x7f69ac0745c0 2024-03-21T13:27:00.478 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.474+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 <== mon.2 v1:172.21.15.43:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 392748637 0 0) 0x7f69ac1a99e0 con 0x7f69ac073ad0 2024-03-21T13:27:00.478 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.474+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 >> v1:172.21.15.155:6789/0 conn(0x7f69ac0745c0 legacy=0x7f69ac1a35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:00.478 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.474+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 >> v1:172.21.15.43:6789/0 conn(0x7f69ac110170 legacy=0x7f69ac1a6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:00.478 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.474+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 --> v1:172.21.15.43:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ac1abe60 con 0x7f69ac073ad0 2024-03-21T13:27:00.478 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.474+0000 7f69b375c640 1 -- 172.21.15.43:0/2048545697 --> v1:172.21.15.43:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69ac1a89d0 con 0x7f69ac073ad0 2024-03-21T13:27:00.478 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.474+0000 7f69b375c640 1 -- 172.21.15.43:0/2048545697 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ac1a8ea0 con 0x7f69ac073ad0 2024-03-21T13:27:00.478 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.475+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 <== mon.2 v1:172.21.15.43:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f69a80015e0 con 0x7f69ac073ad0 2024-03-21T13:27:00.478 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.475+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 <== mon.2 v1:172.21.15.43:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f69a8005c60 con 0x7f69ac073ad0 2024-03-21T13:27:00.478 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.476+0000 7f69b375c640 1 -- 172.21.15.43:0/2048545697 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6990005080 con 0x7f69ac073ad0 2024-03-21T13:27:00.478 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.476+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 <== mon.2 v1:172.21.15.43:6790/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7f69a80038a0 con 0x7f69ac073ad0 2024-03-21T13:27:00.479 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.476+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 <== mon.2 v1:172.21.15.43:6790/0 8 ==== osd_map(822..822 src has 251..822) v4 ==== 7798+0+0 (unknown 1376835242 0 0) 0x7f69a8089d80 con 0x7f69ac073ad0 2024-03-21T13:27:00.479 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.477+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=823}) v3 -- 0x7f69ac1a99e0 con 0x7f69ac073ad0 2024-03-21T13:27:00.480 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.481+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 <== mon.2 v1:172.21.15.43:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f69a8056d90 con 0x7f69ac073ad0 2024-03-21T13:27:00.666 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.666+0000 7f69b375c640 1 -- 172.21.15.43:0/2048545697 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f6990005370 con 0x7f69ac073ad0 2024-03-21T13:27:00.873 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.873+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 <== mon.2 v1:172.21.15.43:6790/0 10 ==== osd_map(823..823 src has 251..823) v4 ==== 628+0+0 (unknown 2613210802 0 0) 0x7f69a8087b70 con 0x7f69ac073ad0 2024-03-21T13:27:00.873 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.873+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=824}) v3 -- 0x7f69ac1a87a0 con 0x7f69ac073ad0 2024-03-21T13:27:00.876 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.876+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 <== mon.2 v1:172.21.15.43:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool dfa6c75f-2043-4860-bf82-1f19a1404959 v823) v1 ==== 217+0+0 (unknown 3234900149 0 0) 0x7f69a805a7e0 con 0x7f69ac073ad0 2024-03-21T13:27:00.989 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:00.989+0000 7f69b375c640 1 -- 172.21.15.43:0/2048545697 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f6990000c40 con 0x7f69ac073ad0 2024-03-21T13:27:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:00 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:00 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2048545697' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T13:27:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:00 smithi155 ceph-mon[33846]: from='client.58841 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T13:27:01.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:00 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:01.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:00 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2048545697' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T13:27:01.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:00 smithi043 ceph-mon[30062]: from='client.58841 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T13:27:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:00 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:00 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2048545697' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T13:27:01.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:00 smithi043 ceph-mon[41699]: from='client.58841 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T13:27:01.878 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.879+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 <== mon.2 v1:172.21.15.43:6790/0 12 ==== osd_map(824..824 src has 251..824) v4 ==== 628+0+0 (unknown 4044773739 0 0) 0x7f69a80526d0 con 0x7f69ac073ad0 2024-03-21T13:27:01.879 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.879+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=825}) v3 -- 0x7f699807c330 con 0x7f69ac073ad0 2024-03-21T13:27:01.883 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.884+0000 7f69a27fc640 1 -- 172.21.15.43:0/2048545697 <== mon.2 v1:172.21.15.43:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool dfa6c75f-2043-4860-bf82-1f19a1404959 v824) v1 ==== 217+0+0 (unknown 1798847595 0 0) 0x7f69a805f0a0 con 0x7f69ac073ad0 2024-03-21T13:27:01.884 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_bytes = 0 for pool dfa6c75f-2043-4860-bf82-1f19a1404959 2024-03-21T13:27:01.887 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.887+0000 7f69b375c640 1 -- 172.21.15.43:0/2048545697 >> v1:172.21.15.43:6800/4199387216 conn(0x7f699806f190 legacy=0x7f6998071650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:01.887 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.887+0000 7f69b375c640 1 -- 172.21.15.43:0/2048545697 >> v1:172.21.15.43:6790/0 conn(0x7f69ac073ad0 legacy=0x7f69ac10e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:01.887 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.887+0000 7f69b375c640 1 -- 172.21.15.43:0/2048545697 shutdown_connections 2024-03-21T13:27:01.887 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.887+0000 7f69b375c640 1 -- 172.21.15.43:0/2048545697 >> 172.21.15.43:0/2048545697 conn(0x7f69ac0fcf80 msgr2=0x7f69ac112630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:01.887 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.887+0000 7f69b375c640 1 -- 172.21.15.43:0/2048545697 shutdown_connections 2024-03-21T13:27:01.887 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.887+0000 7f69b375c640 1 -- 172.21.15.43:0/2048545697 wait complete. 2024-03-21T13:27:01.898 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota dfa6c75f-2043-4860-bf82-1f19a1404959 max_objects 0 2024-03-21T13:27:01.987 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.987+0000 7f227ab55640 1 -- 172.21.15.43:0/1255640999 >> v1:172.21.15.43:6789/0 conn(0x7f227410dff0 legacy=0x7f22741103e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:01.987 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.987+0000 7f227ab55640 1 -- 172.21.15.43:0/1255640999 shutdown_connections 2024-03-21T13:27:01.987 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.987+0000 7f227ab55640 1 -- 172.21.15.43:0/1255640999 >> 172.21.15.43:0/1255640999 conn(0x7f22740fbea0 msgr2=0x7f22740fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:01.987 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.987+0000 7f227ab55640 1 -- 172.21.15.43:0/1255640999 shutdown_connections 2024-03-21T13:27:01.987 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.987+0000 7f227ab55640 1 -- 172.21.15.43:0/1255640999 wait complete. 2024-03-21T13:27:01.988 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.987+0000 7f227ab55640 1 Processor -- start 2024-03-21T13:27:01.988 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.987+0000 7f227ab55640 1 -- start start 2024-03-21T13:27:01.988 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.988+0000 7f227ab55640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f22741a6530 con 0x7f227410dff0 2024-03-21T13:27:01.988 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.988+0000 7f227ab55640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f22741a7770 con 0x7f227410a4e0 2024-03-21T13:27:01.988 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.988+0000 7f227ab55640 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f22741a89b0 con 0x7f22741026e0 2024-03-21T13:27:01.988 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.988+0000 7f22788f1640 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7f22741026e0 0x7f2274109d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:37646/0 (socket says 172.21.15.43:37646) 2024-03-21T13:27:01.988 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.988+0000 7f22788f1640 1 -- 172.21.15.43:0/2837794715 learned_addr learned my addr 172.21.15.43:0/2837794715 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T13:27:01.990 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.988+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4138218121 0 0) 0x7f22741a89b0 con 0x7f22741026e0 2024-03-21T13:27:01.991 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.988+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2264003520 con 0x7f22741026e0 2024-03-21T13:27:01.991 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.988+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1550541604 0 0) 0x7f22741a6530 con 0x7f227410dff0 2024-03-21T13:27:01.991 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.988+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22741a89b0 con 0x7f227410dff0 2024-03-21T13:27:01.991 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.988+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 458777400 0 0) 0x7f22741a7770 con 0x7f227410a4e0 2024-03-21T13:27:01.991 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.988+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22741a6530 con 0x7f227410a4e0 2024-03-21T13:27:01.991 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.988+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 475431096 0 0) 0x7f22741a89b0 con 0x7f227410dff0 2024-03-21T13:27:01.991 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.988+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f22741a7770 con 0x7f227410dff0 2024-03-21T13:27:01.991 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.988+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f226c001d60 con 0x7f227410dff0 2024-03-21T13:27:01.991 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.988+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2500947849 0 0) 0x7f2264003520 con 0x7f22741026e0 2024-03-21T13:27:01.991 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.988+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f22741a89b0 con 0x7f22741026e0 2024-03-21T13:27:01.991 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.988+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f22680018b0 con 0x7f22741026e0 2024-03-21T13:27:01.991 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.988+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1813407872 0 0) 0x7f22741a6530 con 0x7f227410a4e0 2024-03-21T13:27:01.991 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.988+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2264003520 con 0x7f227410a4e0 2024-03-21T13:27:01.992 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.988+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f22600012b0 con 0x7f227410a4e0 2024-03-21T13:27:01.992 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.988+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1654398569 0 0) 0x7f22741a7770 con 0x7f227410dff0 2024-03-21T13:27:01.992 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.988+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 >> v1:172.21.15.43:6790/0 conn(0x7f22741026e0 legacy=0x7f2274109d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:01.992 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.988+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 >> v1:172.21.15.155:6789/0 conn(0x7f227410a4e0 legacy=0x7f22741a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:01.992 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.988+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22741a9bf0 con 0x7f227410dff0 2024-03-21T13:27:01.992 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.988+0000 7f227ab55640 1 -- 172.21.15.43:0/2837794715 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22741a6760 con 0x7f227410dff0 2024-03-21T13:27:01.992 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.989+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f226c004940 con 0x7f227410dff0 2024-03-21T13:27:01.992 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.989+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f226c005210 con 0x7f227410dff0 2024-03-21T13:27:01.992 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.989+0000 7f227ab55640 1 -- 172.21.15.43:0/2837794715 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f22741a6c30 con 0x7f227410dff0 2024-03-21T13:27:01.992 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.990+0000 7f227ab55640 1 -- 172.21.15.43:0/2837794715 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f227406a170 con 0x7f227410dff0 2024-03-21T13:27:01.992 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.990+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7f226c003a70 con 0x7f227410dff0 2024-03-21T13:27:01.992 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.991+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(824..824 src has 251..824) v4 ==== 7798+0+0 (unknown 2403502014 0 0) 0x7f226c088ea0 con 0x7f227410dff0 2024-03-21T13:27:01.992 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.991+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=825}) v3 -- 0x7f22741a7770 con 0x7f227410dff0 2024-03-21T13:27:01.995 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:01.995+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f226c055fb0 con 0x7f227410dff0 2024-03-21T13:27:02.181 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:02.181+0000 7f227ab55640 1 -- 172.21.15.43:0/2837794715 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f227404f500 con 0x7f227410dff0 2024-03-21T13:27:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:01 smithi155 ceph-mon[33846]: pgmap v1795: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:01 smithi155 ceph-mon[33846]: from='client.58841 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]': finished 2024-03-21T13:27:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:01 smithi155 ceph-mon[33846]: osdmap e823: 8 total, 8 up, 8 in 2024-03-21T13:27:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:01 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2048545697' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T13:27:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:01 smithi155 ceph-mon[33846]: from='client.58841 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T13:27:02.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:01 smithi043 ceph-mon[30062]: pgmap v1795: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:01 smithi043 ceph-mon[30062]: from='client.58841 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]': finished 2024-03-21T13:27:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:01 smithi043 ceph-mon[30062]: osdmap e823: 8 total, 8 up, 8 in 2024-03-21T13:27:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:01 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2048545697' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T13:27:02.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:01 smithi043 ceph-mon[30062]: from='client.58841 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T13:27:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:01 smithi043 ceph-mon[41699]: pgmap v1795: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:01 smithi043 ceph-mon[41699]: from='client.58841 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]': finished 2024-03-21T13:27:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:01 smithi043 ceph-mon[41699]: osdmap e823: 8 total, 8 up, 8 in 2024-03-21T13:27:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:01 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2048545697' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T13:27:02.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:01 smithi043 ceph-mon[41699]: from='client.58841 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T13:27:02.883 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:02.883+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool dfa6c75f-2043-4860-bf82-1f19a1404959 v825) v1 ==== 221+0+0 (unknown 3240950884 0 0) 0x7f227404f500 con 0x7f227410dff0 2024-03-21T13:27:02.883 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:02.883+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 <== mon.0 v1:172.21.15.43:6789/0 11 ==== osd_map(825..825 src has 251..825) v4 ==== 628+0+0 (unknown 2356912366 0 0) 0x7f226c051e90 con 0x7f227410dff0 2024-03-21T13:27:02.883 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:02.883+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=826}) v3 -- 0x7f22741a89b0 con 0x7f227410dff0 2024-03-21T13:27:03.000 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.000+0000 7f227ab55640 1 -- 172.21.15.43:0/2837794715 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f2274108300 con 0x7f227410dff0 2024-03-21T13:27:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:02 smithi155 ceph-mon[33846]: from='client.58841 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]': finished 2024-03-21T13:27:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:02 smithi155 ceph-mon[33846]: osdmap e824: 8 total, 8 up, 8 in 2024-03-21T13:27:03.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:02 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2837794715' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T13:27:03.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:02 smithi043 ceph-mon[30062]: from='client.58841 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]': finished 2024-03-21T13:27:03.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:02 smithi043 ceph-mon[30062]: osdmap e824: 8 total, 8 up, 8 in 2024-03-21T13:27:03.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:02 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2837794715' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T13:27:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:02 smithi043 ceph-mon[41699]: from='client.58841 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]': finished 2024-03-21T13:27:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:02 smithi043 ceph-mon[41699]: osdmap e824: 8 total, 8 up, 8 in 2024-03-21T13:27:03.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:02 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2837794715' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T13:27:03.350 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.350+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 <== mon.0 v1:172.21.15.43:6789/0 12 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool dfa6c75f-2043-4860-bf82-1f19a1404959 v826) v1 ==== 221+0+0 (unknown 1144125622 0 0) 0x7f226c059a00 con 0x7f227410dff0 2024-03-21T13:27:03.350 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_objects = 0 for pool dfa6c75f-2043-4860-bf82-1f19a1404959 2024-03-21T13:27:03.350 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.350+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 <== mon.0 v1:172.21.15.43:6789/0 13 ==== osd_map(826..826 src has 251..826) v4 ==== 628+0+0 (unknown 1696360106 0 0) 0x7f226c05de30 con 0x7f227410dff0 2024-03-21T13:27:03.350 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.350+0000 7f2271ffb640 1 -- 172.21.15.43:0/2837794715 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=827}) v3 -- 0x7f22741a6530 con 0x7f227410dff0 2024-03-21T13:27:03.354 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.354+0000 7f227ab55640 1 -- 172.21.15.43:0/2837794715 >> v1:172.21.15.43:6800/4199387216 conn(0x7f226406f070 legacy=0x7f2264071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:03.354 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.354+0000 7f227ab55640 1 -- 172.21.15.43:0/2837794715 >> v1:172.21.15.43:6789/0 conn(0x7f227410dff0 legacy=0x7f22741a4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:03.354 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.354+0000 7f227ab55640 1 -- 172.21.15.43:0/2837794715 shutdown_connections 2024-03-21T13:27:03.354 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.354+0000 7f227ab55640 1 -- 172.21.15.43:0/2837794715 >> 172.21.15.43:0/2837794715 conn(0x7f22740fbea0 msgr2=0x7f22740fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:03.355 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.355+0000 7f227ab55640 1 -- 172.21.15.43:0/2837794715 shutdown_connections 2024-03-21T13:27:03.355 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.355+0000 7f227ab55640 1 -- 172.21.15.43:0/2837794715 wait complete. 2024-03-21T13:27:03.369 INFO:tasks.workunit.client.0.smithi043.stderr:+ wait 159111 2024-03-21T13:27:03.370 INFO:tasks.workunit.client.0.smithi043.stderr:+ '[' 0 -ne 0 ']' 2024-03-21T13:27:03.370 INFO:tasks.workunit.client.0.smithi043.stderr:+ true 2024-03-21T13:27:03.370 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p dfa6c75f-2043-4860-bf82-1f19a1404959 put three /etc/passwd 2024-03-21T13:27:03.406 INFO:tasks.workunit.client.0.smithi043.stderr:++ uuidgen 2024-03-21T13:27:03.406 INFO:tasks.workunit.client.0.smithi043.stderr:+ pp=c3eefea8-5703-4b79-a4c9-c6adb3b164cb 2024-03-21T13:27:03.406 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool create c3eefea8-5703-4b79-a4c9-c6adb3b164cb 12 2024-03-21T13:27:03.494 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.492+0000 7fee5a30f640 1 -- 172.21.15.43:0/3665957573 >> v1:172.21.15.43:6790/0 conn(0x7fee5410a4e0 legacy=0x7fee5410c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:03.494 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.492+0000 7fee5a30f640 1 -- 172.21.15.43:0/3665957573 shutdown_connections 2024-03-21T13:27:03.494 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.492+0000 7fee5a30f640 1 -- 172.21.15.43:0/3665957573 >> 172.21.15.43:0/3665957573 conn(0x7fee540fbea0 msgr2=0x7fee540fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:03.494 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.492+0000 7fee5a30f640 1 -- 172.21.15.43:0/3665957573 shutdown_connections 2024-03-21T13:27:03.494 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.492+0000 7fee5a30f640 1 -- 172.21.15.43:0/3665957573 wait complete. 2024-03-21T13:27:03.494 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.492+0000 7fee5a30f640 1 Processor -- start 2024-03-21T13:27:03.494 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.492+0000 7fee5a30f640 1 -- start start 2024-03-21T13:27:03.494 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee5a30f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fee541a87b0 con 0x7fee541026e0 2024-03-21T13:27:03.494 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee5a30f640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fee541a99f0 con 0x7fee5410a4e0 2024-03-21T13:27:03.495 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee5a30f640 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fee541aac30 con 0x7fee5410dff0 2024-03-21T13:27:03.495 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee53fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fee541026e0 0x7fee54109d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60112/0 (socket says 172.21.15.43:60112) 2024-03-21T13:27:03.495 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee53fff640 1 -- 172.21.15.43:0/3792256861 learned_addr learned my addr 172.21.15.43:0/3792256861 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T13:27:03.495 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2616432378 0 0) 0x7fee541aac30 con 0x7fee5410dff0 2024-03-21T13:27:03.495 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee3c003520 con 0x7fee5410dff0 2024-03-21T13:27:03.495 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3547457203 0 0) 0x7fee541a87b0 con 0x7fee541026e0 2024-03-21T13:27:03.495 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee541aac30 con 0x7fee541026e0 2024-03-21T13:27:03.495 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2415288852 0 0) 0x7fee541a99f0 con 0x7fee5410a4e0 2024-03-21T13:27:03.495 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee541a87b0 con 0x7fee5410a4e0 2024-03-21T13:27:03.495 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1649878574 0 0) 0x7fee3c003520 con 0x7fee5410dff0 2024-03-21T13:27:03.495 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fee541a99f0 con 0x7fee5410dff0 2024-03-21T13:27:03.495 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fee4c001c30 con 0x7fee5410dff0 2024-03-21T13:27:03.495 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1758897432 0 0) 0x7fee541aac30 con 0x7fee541026e0 2024-03-21T13:27:03.495 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fee3c003520 con 0x7fee541026e0 2024-03-21T13:27:03.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fee440012d0 con 0x7fee541026e0 2024-03-21T13:27:03.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1299740518 0 0) 0x7fee541a87b0 con 0x7fee5410a4e0 2024-03-21T13:27:03.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fee541aac30 con 0x7fee5410a4e0 2024-03-21T13:27:03.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fee480043d0 con 0x7fee5410a4e0 2024-03-21T13:27:03.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 <== mon.2 v1:172.21.15.43:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3843783891 0 0) 0x7fee541a99f0 con 0x7fee5410dff0 2024-03-21T13:27:03.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 >> v1:172.21.15.155:6789/0 conn(0x7fee5410a4e0 legacy=0x7fee541a3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:03.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 >> v1:172.21.15.43:6789/0 conn(0x7fee541026e0 legacy=0x7fee54109d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:03.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 --> v1:172.21.15.43:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee541abe70 con 0x7fee5410dff0 2024-03-21T13:27:03.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee5a30f640 1 -- 172.21.15.43:0/3792256861 --> v1:172.21.15.43:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee541a89e0 con 0x7fee5410dff0 2024-03-21T13:27:03.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.493+0000 7fee5a30f640 1 -- 172.21.15.43:0/3792256861 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fee541a8eb0 con 0x7fee5410dff0 2024-03-21T13:27:03.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.494+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 <== mon.2 v1:172.21.15.43:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7fee4c003e20 con 0x7fee5410dff0 2024-03-21T13:27:03.496 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.494+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 <== mon.2 v1:172.21.15.43:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fee4c004d20 con 0x7fee5410dff0 2024-03-21T13:27:03.497 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.495+0000 7fee5a30f640 1 -- 172.21.15.43:0/3792256861 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee40005080 con 0x7fee5410dff0 2024-03-21T13:27:03.497 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.495+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 <== mon.2 v1:172.21.15.43:6790/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7fee4c01b6c0 con 0x7fee5410dff0 2024-03-21T13:27:03.497 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.495+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 <== mon.2 v1:172.21.15.43:6790/0 8 ==== osd_map(826..826 src has 251..826) v4 ==== 7798+0+0 (unknown 2793708130 0 0) 0x7fee4c088e40 con 0x7fee5410dff0 2024-03-21T13:27:03.499 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.500+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 <== mon.2 v1:172.21.15.43:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7fee4c055f70 con 0x7fee5410dff0 2024-03-21T13:27:03.683 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:03.683+0000 7fee5a30f640 1 -- 172.21.15.43:0/3792256861 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pg_num": 12} v 0) v1 -- 0x7fee40005370 con 0x7fee5410dff0 2024-03-21T13:27:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:03 smithi155 ceph-mon[33846]: pgmap v1798: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:27:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:03 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2837794715' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]': finished 2024-03-21T13:27:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:03 smithi155 ceph-mon[33846]: osdmap e825: 8 total, 8 up, 8 in 2024-03-21T13:27:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:03 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2837794715' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T13:27:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:03 smithi155 ceph-mon[33846]: pool 'dfa6c75f-2043-4860-bf82-1f19a1404959' no longer out of quota; removing NO_QUOTA flag 2024-03-21T13:27:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:03 smithi155 ceph-mon[33846]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-21T13:27:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:03 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/2837794715' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]': finished 2024-03-21T13:27:04.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:03 smithi155 ceph-mon[33846]: osdmap e826: 8 total, 8 up, 8 in 2024-03-21T13:27:04.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:03 smithi043 ceph-mon[30062]: pgmap v1798: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:27:04.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:03 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2837794715' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]': finished 2024-03-21T13:27:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:03 smithi043 ceph-mon[30062]: osdmap e825: 8 total, 8 up, 8 in 2024-03-21T13:27:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:03 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2837794715' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T13:27:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:03 smithi043 ceph-mon[30062]: pool 'dfa6c75f-2043-4860-bf82-1f19a1404959' no longer out of quota; removing NO_QUOTA flag 2024-03-21T13:27:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:03 smithi043 ceph-mon[30062]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-21T13:27:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:03 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/2837794715' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]': finished 2024-03-21T13:27:04.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:03 smithi043 ceph-mon[30062]: osdmap e826: 8 total, 8 up, 8 in 2024-03-21T13:27:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:03 smithi043 ceph-mon[41699]: pgmap v1798: 176 pgs: 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-21T13:27:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:03 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2837794715' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]': finished 2024-03-21T13:27:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:03 smithi043 ceph-mon[41699]: osdmap e825: 8 total, 8 up, 8 in 2024-03-21T13:27:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:03 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2837794715' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T13:27:04.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:03 smithi043 ceph-mon[41699]: pool 'dfa6c75f-2043-4860-bf82-1f19a1404959' no longer out of quota; removing NO_QUOTA flag 2024-03-21T13:27:04.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:03 smithi043 ceph-mon[41699]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-21T13:27:04.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:03 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/2837794715' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]': finished 2024-03-21T13:27:04.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:03 smithi043 ceph-mon[41699]: osdmap e826: 8 total, 8 up, 8 in 2024-03-21T13:27:04.356 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.356+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 <== mon.2 v1:172.21.15.43:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pg_num": 12}]=0 pool 'c3eefea8-5703-4b79-a4c9-c6adb3b164cb' created v827) v1 ==== 176+0+0 (unknown 112060173 0 0) 0x7fee4c0599c0 con 0x7fee5410dff0 2024-03-21T13:27:04.469 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.469+0000 7fee5a30f640 1 -- 172.21.15.43:0/3792256861 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pg_num": 12} v 0) v1 -- 0x7fee40001170 con 0x7fee5410dff0 2024-03-21T13:27:04.470 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.470+0000 7fee517fa640 1 -- 172.21.15.43:0/3792256861 <== mon.2 v1:172.21.15.43:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pg_num": 12}]=0 pool 'c3eefea8-5703-4b79-a4c9-c6adb3b164cb' already exists v827) v1 ==== 183+0+0 (unknown 2981915436 0 0) 0x7fee40001170 con 0x7fee5410dff0 2024-03-21T13:27:04.470 INFO:tasks.workunit.client.0.smithi043.stderr:pool 'c3eefea8-5703-4b79-a4c9-c6adb3b164cb' already exists 2024-03-21T13:27:04.472 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.472+0000 7fee5a30f640 1 -- 172.21.15.43:0/3792256861 >> v1:172.21.15.43:6800/4199387216 conn(0x7fee3c06f040 legacy=0x7fee3c071500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:04.472 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.472+0000 7fee5a30f640 1 -- 172.21.15.43:0/3792256861 >> v1:172.21.15.43:6790/0 conn(0x7fee5410dff0 legacy=0x7fee541a6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:04.472 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.472+0000 7fee5a30f640 1 -- 172.21.15.43:0/3792256861 shutdown_connections 2024-03-21T13:27:04.472 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.472+0000 7fee5a30f640 1 -- 172.21.15.43:0/3792256861 >> 172.21.15.43:0/3792256861 conn(0x7fee540fbea0 msgr2=0x7fee540fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:04.472 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.472+0000 7fee5a30f640 1 -- 172.21.15.43:0/3792256861 shutdown_connections 2024-03-21T13:27:04.472 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.472+0000 7fee5a30f640 1 -- 172.21.15.43:0/3792256861 wait complete. 2024-03-21T13:27:04.481 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool application enable c3eefea8-5703-4b79-a4c9-c6adb3b164cb rados 2024-03-21T13:27:04.564 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.564+0000 7f738cc4e640 1 -- 172.21.15.43:0/1790213982 >> v1:172.21.15.43:6790/0 conn(0x7f738810a360 legacy=0x7f738810c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:04.564 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.564+0000 7f738cc4e640 1 -- 172.21.15.43:0/1790213982 shutdown_connections 2024-03-21T13:27:04.564 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.564+0000 7f738cc4e640 1 -- 172.21.15.43:0/1790213982 >> 172.21.15.43:0/1790213982 conn(0x7f73880fcfd0 msgr2=0x7f73880ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:04.564 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.564+0000 7f738cc4e640 1 -- 172.21.15.43:0/1790213982 shutdown_connections 2024-03-21T13:27:04.564 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.564+0000 7f738cc4e640 1 -- 172.21.15.43:0/1790213982 wait complete. 2024-03-21T13:27:04.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.565+0000 7f738cc4e640 1 Processor -- start 2024-03-21T13:27:04.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.565+0000 7f738cc4e640 1 -- start start 2024-03-21T13:27:04.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.565+0000 7f738cc4e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f73881a87d0 con 0x7f738810df60 2024-03-21T13:27:04.567 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.565+0000 7f738cc4e640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f73881a99f0 con 0x7f73881075c0 2024-03-21T13:27:04.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.565+0000 7f738cc4e640 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f73881aac30 con 0x7f738810a360 2024-03-21T13:27:04.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.565+0000 7f7386d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f738810df60 0x7f73881a6ef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60134/0 (socket says 172.21.15.43:60134) 2024-03-21T13:27:04.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.565+0000 7f7385d9b640 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7f738810a360 0x7f73881a37c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:37720/0 (socket says 172.21.15.43:37720) 2024-03-21T13:27:04.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.565+0000 7f7386d9d640 1 -- 172.21.15.43:0/763440896 learned_addr learned my addr 172.21.15.43:0/763440896 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T13:27:04.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.565+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1531050008 0 0) 0x7f73881aac30 con 0x7f738810a360 2024-03-21T13:27:04.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.565+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7370003160 con 0x7f738810a360 2024-03-21T13:27:04.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.565+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 177728396 0 0) 0x7f73881a87d0 con 0x7f738810df60 2024-03-21T13:27:04.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.565+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73881aac30 con 0x7f738810df60 2024-03-21T13:27:04.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.566+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972760999 0 0) 0x7f73881a99f0 con 0x7f73881075c0 2024-03-21T13:27:04.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.566+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73881a87d0 con 0x7f73881075c0 2024-03-21T13:27:04.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.566+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 427950548 0 0) 0x7f7370003160 con 0x7f738810a360 2024-03-21T13:27:04.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.566+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f73881a99f0 con 0x7f738810a360 2024-03-21T13:27:04.568 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.566+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f737c004300 con 0x7f738810a360 2024-03-21T13:27:04.569 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.566+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2502834118 0 0) 0x7f73881aac30 con 0x7f738810df60 2024-03-21T13:27:04.569 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.566+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7370003160 con 0x7f738810df60 2024-03-21T13:27:04.569 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.566+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f73800015b0 con 0x7f738810df60 2024-03-21T13:27:04.569 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.566+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1002582278 0 0) 0x7f73881a87d0 con 0x7f73881075c0 2024-03-21T13:27:04.569 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.566+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f73881aac30 con 0x7f73881075c0 2024-03-21T13:27:04.569 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.566+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f7378001830 con 0x7f73881075c0 2024-03-21T13:27:04.569 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.566+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 <== mon.2 v1:172.21.15.43:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2580629617 0 0) 0x7f73881a99f0 con 0x7f738810a360 2024-03-21T13:27:04.569 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.566+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 >> v1:172.21.15.155:6789/0 conn(0x7f73881075c0 legacy=0x7f7388106740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:04.569 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.566+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 >> v1:172.21.15.43:6789/0 conn(0x7f738810df60 legacy=0x7f73881a6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:04.569 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.566+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 --> v1:172.21.15.43:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73881abe70 con 0x7f738810a360 2024-03-21T13:27:04.569 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.566+0000 7f738cc4e640 1 -- 172.21.15.43:0/763440896 --> v1:172.21.15.43:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73881aae60 con 0x7f738810a360 2024-03-21T13:27:04.569 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.566+0000 7f738cc4e640 1 -- 172.21.15.43:0/763440896 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f73881ab330 con 0x7f738810a360 2024-03-21T13:27:04.569 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.566+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 <== mon.2 v1:172.21.15.43:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f737c0044a0 con 0x7f738810a360 2024-03-21T13:27:04.569 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.566+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 <== mon.2 v1:172.21.15.43:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f737c004bf0 con 0x7f738810a360 2024-03-21T13:27:04.570 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.567+0000 7f738cc4e640 1 -- 172.21.15.43:0/763440896 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7368005080 con 0x7f738810a360 2024-03-21T13:27:04.570 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.567+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 <== mon.2 v1:172.21.15.43:6790/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7f737c001a30 con 0x7f738810a360 2024-03-21T13:27:04.570 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.568+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 <== mon.2 v1:172.21.15.43:6790/0 8 ==== osd_map(827..827 src has 251..827) v4 ==== 8173+0+0 (unknown 1187819435 0 0) 0x7f737c089df0 con 0x7f738810a360 2024-03-21T13:27:04.572 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.572+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 <== mon.2 v1:172.21.15.43:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f737c004bd0 con 0x7f738810a360 2024-03-21T13:27:04.758 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:04.758+0000 7f738cc4e640 1 -- 172.21.15.43:0/763440896 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "app": "rados"} v 0) v1 -- 0x7f7368005370 con 0x7f738810a360 2024-03-21T13:27:05.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3792256861' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pg_num": 12}]: dispatch 2024-03-21T13:27:05.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:04 smithi155 ceph-mon[33846]: from='client.58862 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pg_num": 12}]: dispatch 2024-03-21T13:27:05.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:04 smithi155 ceph-mon[33846]: from='client.58862 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pg_num": 12}]': finished 2024-03-21T13:27:05.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:04 smithi155 ceph-mon[33846]: osdmap e827: 8 total, 8 up, 8 in 2024-03-21T13:27:05.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3792256861' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pg_num": 12}]: dispatch 2024-03-21T13:27:05.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:04 smithi155 ceph-mon[33846]: from='client.58862 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pg_num": 12}]: dispatch 2024-03-21T13:27:05.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:04 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/763440896' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "app": "rados"}]: dispatch 2024-03-21T13:27:05.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:04 smithi155 ceph-mon[33846]: from='client.58868 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "app": "rados"}]: dispatch 2024-03-21T13:27:05.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3792256861' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pg_num": 12}]: dispatch 2024-03-21T13:27:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:04 smithi043 ceph-mon[30062]: from='client.58862 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pg_num": 12}]: dispatch 2024-03-21T13:27:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:04 smithi043 ceph-mon[30062]: from='client.58862 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pg_num": 12}]': finished 2024-03-21T13:27:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:04 smithi043 ceph-mon[30062]: osdmap e827: 8 total, 8 up, 8 in 2024-03-21T13:27:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3792256861' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pg_num": 12}]: dispatch 2024-03-21T13:27:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:04 smithi043 ceph-mon[30062]: from='client.58862 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pg_num": 12}]: dispatch 2024-03-21T13:27:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:04 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/763440896' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "app": "rados"}]: dispatch 2024-03-21T13:27:05.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:04 smithi043 ceph-mon[30062]: from='client.58868 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "app": "rados"}]: dispatch 2024-03-21T13:27:05.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3792256861' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pg_num": 12}]: dispatch 2024-03-21T13:27:05.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:04 smithi043 ceph-mon[41699]: from='client.58862 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pg_num": 12}]: dispatch 2024-03-21T13:27:05.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:04 smithi043 ceph-mon[41699]: from='client.58862 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pg_num": 12}]': finished 2024-03-21T13:27:05.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:04 smithi043 ceph-mon[41699]: osdmap e827: 8 total, 8 up, 8 in 2024-03-21T13:27:05.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3792256861' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pg_num": 12}]: dispatch 2024-03-21T13:27:05.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:04 smithi043 ceph-mon[41699]: from='client.58862 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pg_num": 12}]: dispatch 2024-03-21T13:27:05.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:04 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/763440896' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "app": "rados"}]: dispatch 2024-03-21T13:27:05.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:04 smithi043 ceph-mon[41699]: from='client.58868 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "app": "rados"}]: dispatch 2024-03-21T13:27:05.362 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:05.362+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 <== mon.2 v1:172.21.15.43:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "app": "rados"}]=0 enabled application 'rados' on pool 'c3eefea8-5703-4b79-a4c9-c6adb3b164cb' v828) v1 ==== 213+0+0 (unknown 1806291798 0 0) 0x7f737c056d90 con 0x7f738810a360 2024-03-21T13:27:05.484 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:05.484+0000 7f738cc4e640 1 -- 172.21.15.43:0/763440896 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "app": "rados"} v 0) v1 -- 0x7f7368005b80 con 0x7f738810a360 2024-03-21T13:27:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:05 smithi155 ceph-mon[33846]: pgmap v1802: 188 pgs: 12 unknown, 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-21T13:27:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:05 smithi155 ceph-mon[33846]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:27:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:05 smithi155 ceph-mon[33846]: from='client.58868 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "app": "rados"}]': finished 2024-03-21T13:27:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:05 smithi155 ceph-mon[33846]: osdmap e828: 8 total, 8 up, 8 in 2024-03-21T13:27:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:05 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/763440896' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "app": "rados"}]: dispatch 2024-03-21T13:27:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:05 smithi155 ceph-mon[33846]: from='client.58868 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "app": "rados"}]: dispatch 2024-03-21T13:27:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:05 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:27:06.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:05 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:27:06.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:05 smithi043 ceph-mon[30062]: pgmap v1802: 188 pgs: 12 unknown, 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-21T13:27:06.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:05 smithi043 ceph-mon[30062]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:27:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:05 smithi043 ceph-mon[30062]: from='client.58868 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "app": "rados"}]': finished 2024-03-21T13:27:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:05 smithi043 ceph-mon[30062]: osdmap e828: 8 total, 8 up, 8 in 2024-03-21T13:27:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:05 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/763440896' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "app": "rados"}]: dispatch 2024-03-21T13:27:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:05 smithi043 ceph-mon[30062]: from='client.58868 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "app": "rados"}]: dispatch 2024-03-21T13:27:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:05 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:27:06.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:05 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:27:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:05 smithi043 ceph-mon[41699]: pgmap v1802: 188 pgs: 12 unknown, 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-21T13:27:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:05 smithi043 ceph-mon[41699]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:27:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:05 smithi043 ceph-mon[41699]: from='client.58868 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "app": "rados"}]': finished 2024-03-21T13:27:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:05 smithi043 ceph-mon[41699]: osdmap e828: 8 total, 8 up, 8 in 2024-03-21T13:27:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:05 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/763440896' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "app": "rados"}]: dispatch 2024-03-21T13:27:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:05 smithi043 ceph-mon[41699]: from='client.58868 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "app": "rados"}]: dispatch 2024-03-21T13:27:06.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:05 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:27:06.235 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:05 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:27:06.363 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.364+0000 7f73777fe640 1 -- 172.21.15.43:0/763440896 <== mon.2 v1:172.21.15.43:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "app": "rados"}]=0 enabled application 'rados' on pool 'c3eefea8-5703-4b79-a4c9-c6adb3b164cb' v829) v1 ==== 213+0+0 (unknown 1367877040 0 0) 0x7f737c056d90 con 0x7f738810a360 2024-03-21T13:27:06.364 INFO:tasks.workunit.client.0.smithi043.stderr:enabled application 'rados' on pool 'c3eefea8-5703-4b79-a4c9-c6adb3b164cb' 2024-03-21T13:27:06.366 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.366+0000 7f738cc4e640 1 -- 172.21.15.43:0/763440896 >> v1:172.21.15.43:6800/4199387216 conn(0x7f737006f0a0 legacy=0x7f7370071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:06.366 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.366+0000 7f738cc4e640 1 -- 172.21.15.43:0/763440896 >> v1:172.21.15.43:6790/0 conn(0x7f738810a360 legacy=0x7f73881a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:06.366 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.367+0000 7f738cc4e640 1 -- 172.21.15.43:0/763440896 shutdown_connections 2024-03-21T13:27:06.366 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.367+0000 7f738cc4e640 1 -- 172.21.15.43:0/763440896 >> 172.21.15.43:0/763440896 conn(0x7f73880fcfd0 msgr2=0x7f73880738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:06.366 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.367+0000 7f738cc4e640 1 -- 172.21.15.43:0/763440896 shutdown_connections 2024-03-21T13:27:06.367 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.367+0000 7f738cc4e640 1 -- 172.21.15.43:0/763440896 wait complete. 2024-03-21T13:27:06.378 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota c3eefea8-5703-4b79-a4c9-c6adb3b164cb max_objects 10 2024-03-21T13:27:06.467 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.466+0000 7fc6ae484640 1 -- 172.21.15.43:0/3154613265 >> v1:172.21.15.43:6789/0 conn(0x7fc6a81075b0 legacy=0x7fc6a8107990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:06.467 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.466+0000 7fc6ae484640 1 -- 172.21.15.43:0/3154613265 shutdown_connections 2024-03-21T13:27:06.467 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.466+0000 7fc6ae484640 1 -- 172.21.15.43:0/3154613265 >> 172.21.15.43:0/3154613265 conn(0x7fc6a80fcf80 msgr2=0x7fc6a80ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:06.467 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.467+0000 7fc6ae484640 1 -- 172.21.15.43:0/3154613265 shutdown_connections 2024-03-21T13:27:06.467 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.467+0000 7fc6ae484640 1 -- 172.21.15.43:0/3154613265 wait complete. 2024-03-21T13:27:06.467 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.467+0000 7fc6ae484640 1 Processor -- start 2024-03-21T13:27:06.467 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.467+0000 7fc6ae484640 1 -- start start 2024-03-21T13:27:06.467 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.467+0000 7fc6ae484640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc6a81a6570 con 0x7fc6a810df50 2024-03-21T13:27:06.468 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.467+0000 7fc6ae484640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc6a81a77b0 con 0x7fc6a810a350 2024-03-21T13:27:06.468 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.467+0000 7fc6ae484640 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc6a81a89f0 con 0x7fc6a81075b0 2024-03-21T13:27:06.468 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.468+0000 7fc6a7fff640 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7fc6a81075b0 0x7fc6a8106930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:37736/0 (socket says 172.21.15.43:37736) 2024-03-21T13:27:06.468 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.468+0000 7fc6a7fff640 1 -- 172.21.15.43:0/358567297 learned_addr learned my addr 172.21.15.43:0/358567297 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T13:27:06.470 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.468+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1439509025 0 0) 0x7fc6a81a89f0 con 0x7fc6a81075b0 2024-03-21T13:27:06.470 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.468+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc690003160 con 0x7fc6a81075b0 2024-03-21T13:27:06.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.468+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 776669072 0 0) 0x7fc6a81a6570 con 0x7fc6a810df50 2024-03-21T13:27:06.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.468+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6a81a89f0 con 0x7fc6a810df50 2024-03-21T13:27:06.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.468+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2495309504 0 0) 0x7fc6a81a77b0 con 0x7fc6a810a350 2024-03-21T13:27:06.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.468+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6a81a6570 con 0x7fc6a810a350 2024-03-21T13:27:06.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.468+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1907024838 0 0) 0x7fc690003160 con 0x7fc6a81075b0 2024-03-21T13:27:06.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.468+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc6a81a77b0 con 0x7fc6a81075b0 2024-03-21T13:27:06.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.468+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fc698004330 con 0x7fc6a81075b0 2024-03-21T13:27:06.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.468+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1645941250 0 0) 0x7fc6a81a89f0 con 0x7fc6a810df50 2024-03-21T13:27:06.471 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.468+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc690003160 con 0x7fc6a810df50 2024-03-21T13:27:06.472 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.468+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fc6a0001c10 con 0x7fc6a810df50 2024-03-21T13:27:06.472 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.468+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3546002070 0 0) 0x7fc6a81a6570 con 0x7fc6a810a350 2024-03-21T13:27:06.472 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.468+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc6a81a89f0 con 0x7fc6a810a350 2024-03-21T13:27:06.472 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.468+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fc69c001830 con 0x7fc6a810a350 2024-03-21T13:27:06.472 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.468+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 <== mon.2 v1:172.21.15.43:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1700460069 0 0) 0x7fc6a81a77b0 con 0x7fc6a81075b0 2024-03-21T13:27:06.472 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.468+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 >> v1:172.21.15.155:6789/0 conn(0x7fc6a810a350 legacy=0x7fc6a81a13c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:06.472 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.468+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 >> v1:172.21.15.43:6789/0 conn(0x7fc6a810df50 legacy=0x7fc6a81a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:06.472 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.468+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 --> v1:172.21.15.43:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a81a9c30 con 0x7fc6a81075b0 2024-03-21T13:27:06.473 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.468+0000 7fc6ae484640 1 -- 172.21.15.43:0/358567297 --> v1:172.21.15.43:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a81a67a0 con 0x7fc6a81075b0 2024-03-21T13:27:06.473 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.468+0000 7fc6ae484640 1 -- 172.21.15.43:0/358567297 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a81a6c70 con 0x7fc6a81075b0 2024-03-21T13:27:06.473 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.469+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 <== mon.2 v1:172.21.15.43:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7fc698001a60 con 0x7fc6a81075b0 2024-03-21T13:27:06.473 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.469+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 <== mon.2 v1:172.21.15.43:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fc698004d80 con 0x7fc6a81075b0 2024-03-21T13:27:06.473 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.469+0000 7fc6ae484640 1 -- 172.21.15.43:0/358567297 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc694005080 con 0x7fc6a81075b0 2024-03-21T13:27:06.473 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.470+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 <== mon.2 v1:172.21.15.43:6790/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7fc698004d60 con 0x7fc6a81075b0 2024-03-21T13:27:06.473 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.470+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 <== mon.2 v1:172.21.15.43:6790/0 8 ==== osd_map(829..829 src has 251..829) v4 ==== 8186+0+0 (unknown 1793609786 0 0) 0x7fc698089fe0 con 0x7fc6a81075b0 2024-03-21T13:27:06.474 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.475+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 <== mon.2 v1:172.21.15.43:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7fc698056e30 con 0x7fc6a81075b0 2024-03-21T13:27:06.662 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:06.662+0000 7fc6ae484640 1 -- 172.21.15.43:0/358567297 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7fc694005370 con 0x7fc6a81075b0 2024-03-21T13:27:07.368 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:07.369+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 <== mon.2 v1:172.21.15.43:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool c3eefea8-5703-4b79-a4c9-c6adb3b164cb v830) v1 ==== 223+0+0 (unknown 2678231520 0 0) 0x7fc69805a880 con 0x7fc6a81075b0 2024-03-21T13:27:07.484 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:07.484+0000 7fc6ae484640 1 -- 172.21.15.43:0/358567297 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7fc694001090 con 0x7fc6a81075b0 2024-03-21T13:27:07.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:07 smithi155 ceph-mon[33846]: from='client.58868 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "app": "rados"}]': finished 2024-03-21T13:27:07.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:07 smithi155 ceph-mon[33846]: osdmap e829: 8 total, 8 up, 8 in 2024-03-21T13:27:07.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:07 smithi155 ceph-mon[33846]: pgmap v1805: 188 pgs: 12 unknown, 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-21T13:27:07.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:07 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/358567297' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:27:07.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:07 smithi155 ceph-mon[33846]: from='client.58874 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:27:07.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:07 smithi043 ceph-mon[30062]: from='client.58868 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "app": "rados"}]': finished 2024-03-21T13:27:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:07 smithi043 ceph-mon[30062]: osdmap e829: 8 total, 8 up, 8 in 2024-03-21T13:27:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:07 smithi043 ceph-mon[30062]: pgmap v1805: 188 pgs: 12 unknown, 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-21T13:27:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:07 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/358567297' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:27:07.734 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:07 smithi043 ceph-mon[30062]: from='client.58874 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:27:07.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:07 smithi043 ceph-mon[41699]: from='client.58868 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "app": "rados"}]': finished 2024-03-21T13:27:07.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:07 smithi043 ceph-mon[41699]: osdmap e829: 8 total, 8 up, 8 in 2024-03-21T13:27:07.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:07 smithi043 ceph-mon[41699]: pgmap v1805: 188 pgs: 12 unknown, 176 active+clean; 721 KiB data, 873 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-03-21T13:27:07.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:07 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/358567297' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:27:07.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:07 smithi043 ceph-mon[41699]: from='client.58874 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:27:08.371 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:08.371+0000 7fc6a57fa640 1 -- 172.21.15.43:0/358567297 <== mon.2 v1:172.21.15.43:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool c3eefea8-5703-4b79-a4c9-c6adb3b164cb v831) v1 ==== 223+0+0 (unknown 3605476496 0 0) 0x7fc69805f140 con 0x7fc6a81075b0 2024-03-21T13:27:08.371 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_objects = 10 for pool c3eefea8-5703-4b79-a4c9-c6adb3b164cb 2024-03-21T13:27:08.372 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:27:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:27:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:27:08.374 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:08.374+0000 7fc6ae484640 1 -- 172.21.15.43:0/358567297 >> v1:172.21.15.43:6800/4199387216 conn(0x7fc69006f1a0 legacy=0x7fc690071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:08.374 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:08.374+0000 7fc6ae484640 1 -- 172.21.15.43:0/358567297 >> v1:172.21.15.43:6790/0 conn(0x7fc6a81075b0 legacy=0x7fc6a8106930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:27:08.374 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:08.375+0000 7fc6ae484640 1 -- 172.21.15.43:0/358567297 shutdown_connections 2024-03-21T13:27:08.375 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:08.375+0000 7fc6ae484640 1 -- 172.21.15.43:0/358567297 >> 172.21.15.43:0/358567297 conn(0x7fc6a80fcf80 msgr2=0x7fc6a8110c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:27:08.375 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:08.375+0000 7fc6ae484640 1 -- 172.21.15.43:0/358567297 shutdown_connections 2024-03-21T13:27:08.375 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:27:08.375+0000 7fc6ae484640 1 -- 172.21.15.43:0/358567297 wait complete. 2024-03-21T13:27:08.387 INFO:tasks.workunit.client.0.smithi043.stderr:+ sleep 30 2024-03-21T13:27:08.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:08 smithi043 ceph-mon[30062]: from='client.58874 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "field": "max_objects", "val": "10"}]': finished 2024-03-21T13:27:08.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:08 smithi043 ceph-mon[30062]: osdmap e830: 8 total, 8 up, 8 in 2024-03-21T13:27:08.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:08 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/358567297' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:27:08.647 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:08 smithi043 ceph-mon[30062]: from='client.58874 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:27:08.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:08 smithi043 ceph-mon[41699]: from='client.58874 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "field": "max_objects", "val": "10"}]': finished 2024-03-21T13:27:08.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:08 smithi043 ceph-mon[41699]: osdmap e830: 8 total, 8 up, 8 in 2024-03-21T13:27:08.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:08 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/358567297' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:27:08.648 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:08 smithi043 ceph-mon[41699]: from='client.58874 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:27:08.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:08 smithi155 ceph-mon[33846]: from='client.58874 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "field": "max_objects", "val": "10"}]': finished 2024-03-21T13:27:08.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:08 smithi155 ceph-mon[33846]: osdmap e830: 8 total, 8 up, 8 in 2024-03-21T13:27:08.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:08 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/358567297' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:27:08.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:08 smithi155 ceph-mon[33846]: from='client.58874 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "field": "max_objects", "val": "10"}]: dispatch 2024-03-21T13:27:09.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:09 smithi043 ceph-mon[30062]: from='client.58874 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "field": "max_objects", "val": "10"}]': finished 2024-03-21T13:27:09.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:09 smithi043 ceph-mon[30062]: osdmap e831: 8 total, 8 up, 8 in 2024-03-21T13:27:09.630 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:09 smithi043 ceph-mon[30062]: pgmap v1808: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-03-21T13:27:09.630 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:09 smithi043 ceph-mon[41699]: from='client.58874 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "field": "max_objects", "val": "10"}]': finished 2024-03-21T13:27:09.630 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:09 smithi043 ceph-mon[41699]: osdmap e831: 8 total, 8 up, 8 in 2024-03-21T13:27:09.630 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:09 smithi043 ceph-mon[41699]: pgmap v1808: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-03-21T13:27:09.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:09 smithi155 ceph-mon[33846]: from='client.58874 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "field": "max_objects", "val": "10"}]': finished 2024-03-21T13:27:09.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:09 smithi155 ceph-mon[33846]: osdmap e831: 8 total, 8 up, 8 in 2024-03-21T13:27:09.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:09 smithi155 ceph-mon[33846]: pgmap v1808: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-03-21T13:27:10.379 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:27:09 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:27:10.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:10 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:10.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:10 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:10.733 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:10 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:11.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:11 smithi155 ceph-mon[33846]: pgmap v1809: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 972 B/s rd, 4.6 KiB/s wr, 1 op/s 2024-03-21T13:27:11.733 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:11 smithi043 ceph-mon[30062]: pgmap v1809: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 972 B/s rd, 4.6 KiB/s wr, 1 op/s 2024-03-21T13:27:11.734 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:11 smithi043 ceph-mon[41699]: pgmap v1809: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 972 B/s rd, 4.6 KiB/s wr, 1 op/s 2024-03-21T13:27:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:13 smithi155 ceph-mon[33846]: pgmap v1810: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 817 B/s rd, 3.8 KiB/s wr, 1 op/s 2024-03-21T13:27:13.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:13 smithi155 ceph-mon[33846]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:27:13.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:13 smithi043 ceph-mon[30062]: pgmap v1810: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 817 B/s rd, 3.8 KiB/s wr, 1 op/s 2024-03-21T13:27:13.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:13 smithi043 ceph-mon[30062]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:27:13.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:13 smithi043 ceph-mon[41699]: pgmap v1810: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 817 B/s rd, 3.8 KiB/s wr, 1 op/s 2024-03-21T13:27:13.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:13 smithi043 ceph-mon[41699]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T13:27:15.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:15 smithi155 ceph-mon[33846]: pgmap v1811: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-21T13:27:15.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:15 smithi043 ceph-mon[30062]: pgmap v1811: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-21T13:27:15.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:15 smithi043 ceph-mon[41699]: pgmap v1811: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-21T13:27:17.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:17 smithi155 ceph-mon[33846]: pgmap v1812: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.6 KiB/s wr, 1 op/s 2024-03-21T13:27:17.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:17 smithi043 ceph-mon[30062]: pgmap v1812: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.6 KiB/s wr, 1 op/s 2024-03-21T13:27:17.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:17 smithi043 ceph-mon[41699]: pgmap v1812: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.6 KiB/s wr, 1 op/s 2024-03-21T13:27:18.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:27:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:27:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:27:19.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:19 smithi155 ceph-mon[33846]: pgmap v1813: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 998 B/s rd, 0 op/s 2024-03-21T13:27:19.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:19 smithi043 ceph-mon[30062]: pgmap v1813: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 998 B/s rd, 0 op/s 2024-03-21T13:27:19.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:19 smithi043 ceph-mon[41699]: pgmap v1813: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 998 B/s rd, 0 op/s 2024-03-21T13:27:20.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:27:19 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:27:20.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:20 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:20.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:20 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:20.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:20 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:21.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:21 smithi155 ceph-mon[33846]: pgmap v1814: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:21.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:21 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:27:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:21 smithi043 ceph-mon[30062]: pgmap v1814: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:21.984 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:21 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:27:21.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:21 smithi043 ceph-mon[41699]: pgmap v1814: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:21.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:21 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:27:23.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:23 smithi155 ceph-mon[33846]: pgmap v1815: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:23.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:23 smithi043 ceph-mon[30062]: pgmap v1815: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:23.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:23 smithi043 ceph-mon[41699]: pgmap v1815: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:25.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:25 smithi155 ceph-mon[33846]: pgmap v1816: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:25.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:25 smithi043 ceph-mon[30062]: pgmap v1816: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:25.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:25 smithi043 ceph-mon[41699]: pgmap v1816: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:27.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:27 smithi155 ceph-mon[33846]: pgmap v1817: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:27.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:27 smithi043 ceph-mon[30062]: pgmap v1817: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:27.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:27 smithi043 ceph-mon[41699]: pgmap v1817: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:28.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:27:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:27:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:27:29.978 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:29 smithi155 ceph-mon[33846]: pgmap v1818: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:29.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:29 smithi043 ceph-mon[30062]: pgmap v1818: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:29.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:29 smithi043 ceph-mon[41699]: pgmap v1818: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:30.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:27:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:27:30.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:30 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:30.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:30 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:31.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:30 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:31.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:31 smithi043 ceph-mon[30062]: pgmap v1819: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:31.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:31 smithi043 ceph-mon[41699]: pgmap v1819: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:32.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:31 smithi155 ceph-mon[33846]: pgmap v1819: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:33.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:33 smithi043 ceph-mon[30062]: pgmap v1820: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:33.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:33 smithi043 ceph-mon[41699]: pgmap v1820: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:34.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:33 smithi155 ceph-mon[33846]: pgmap v1820: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:35.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:35 smithi043 ceph-mon[30062]: pgmap v1821: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:35.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:35 smithi043 ceph-mon[41699]: pgmap v1821: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:36.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:35 smithi155 ceph-mon[33846]: pgmap v1821: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:36.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:36 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:27:36.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:36 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:27:37.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:36 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:27:37.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:37 smithi043 ceph-mon[30062]: pgmap v1822: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:27:37.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:37 smithi043 ceph-mon[41699]: pgmap v1822: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:27:38.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:37 smithi155 ceph-mon[33846]: pgmap v1822: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:27:38.389 INFO:tasks.workunit.client.0.smithi043.stderr:++ seq 1 10 2024-03-21T13:27:38.389 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-03-21T13:27:38.390 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p c3eefea8-5703-4b79-a4c9-c6adb3b164cb put obj1 /etc/passwd 2024-03-21T13:27:38.429 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-03-21T13:27:38.429 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p c3eefea8-5703-4b79-a4c9-c6adb3b164cb put obj2 /etc/passwd 2024-03-21T13:27:38.465 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-03-21T13:27:38.465 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p c3eefea8-5703-4b79-a4c9-c6adb3b164cb put obj3 /etc/passwd 2024-03-21T13:27:38.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:27:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:27:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:27:38.500 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-03-21T13:27:38.500 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p c3eefea8-5703-4b79-a4c9-c6adb3b164cb put obj4 /etc/passwd 2024-03-21T13:27:38.534 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-03-21T13:27:38.534 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p c3eefea8-5703-4b79-a4c9-c6adb3b164cb put obj5 /etc/passwd 2024-03-21T13:27:38.566 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-03-21T13:27:38.567 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p c3eefea8-5703-4b79-a4c9-c6adb3b164cb put obj6 /etc/passwd 2024-03-21T13:27:38.599 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-03-21T13:27:38.599 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p c3eefea8-5703-4b79-a4c9-c6adb3b164cb put obj7 /etc/passwd 2024-03-21T13:27:38.632 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-03-21T13:27:38.632 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p c3eefea8-5703-4b79-a4c9-c6adb3b164cb put obj8 /etc/passwd 2024-03-21T13:27:38.665 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-03-21T13:27:38.665 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p c3eefea8-5703-4b79-a4c9-c6adb3b164cb put obj9 /etc/passwd 2024-03-21T13:27:38.697 INFO:tasks.workunit.client.0.smithi043.stderr:+ for f in `seq 1 10` 2024-03-21T13:27:38.697 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p c3eefea8-5703-4b79-a4c9-c6adb3b164cb put obj10 /etc/passwd 2024-03-21T13:27:38.731 INFO:tasks.workunit.client.0.smithi043.stderr:+ sleep 30 2024-03-21T13:27:39.981 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:39 smithi155 ceph-mon[33846]: pgmap v1823: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:39.983 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:39 smithi043 ceph-mon[30062]: pgmap v1823: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:39.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:39 smithi043 ceph-mon[41699]: pgmap v1823: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:27:40.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:27:39 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:27:41.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:40 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:41.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:40 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:41.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:40 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:41 smithi155 ceph-mon[33846]: pgmap v1824: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:41 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:27:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:41 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:27:42.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:41 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:27:42.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:41 smithi043 ceph-mon[30062]: pgmap v1824: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:41 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:27:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:41 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:27:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:41 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:27:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:41 smithi043 ceph-mon[41699]: pgmap v1824: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:41 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:27:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:41 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:27:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:41 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:27:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:42 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:27:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:42 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:27:43.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:42 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:27:43.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:42 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:27:43.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:42 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:27:43.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:42 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:27:43.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:42 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:27:43.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:42 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:27:43.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:42 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:27:44.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:43 smithi155 ceph-mon[33846]: pgmap v1825: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:44.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:43 smithi043 ceph-mon[30062]: pgmap v1825: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:44.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:43 smithi043 ceph-mon[41699]: pgmap v1825: 188 pgs: 188 active+clean; 744 KiB data, 878 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:27:46.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:45 smithi155 ceph-mon[33846]: pgmap v1826: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-03-21T13:27:46.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:45 smithi043 ceph-mon[30062]: pgmap v1826: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-03-21T13:27:46.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:45 smithi043 ceph-mon[41699]: pgmap v1826: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-03-21T13:27:48.024 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:47 smithi043 ceph-mon[30062]: pgmap v1827: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-03-21T13:27:48.050 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:47 smithi043 ceph-mon[41699]: pgmap v1827: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-03-21T13:27:48.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:47 smithi155 ceph-mon[33846]: pgmap v1827: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 10 KiB/s wr, 1 op/s 2024-03-21T13:27:48.483 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:27:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:27:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:27:49.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:48 smithi155 ceph-mon[33846]: pool 'c3eefea8-5703-4b79-a4c9-c6adb3b164cb' is full (reached quota's max_objects: 10) 2024-03-21T13:27:49.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:48 smithi155 ceph-mon[33846]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-21T13:27:49.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:48 smithi155 ceph-mon[33846]: osdmap e832: 8 total, 8 up, 8 in 2024-03-21T13:27:49.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:48 smithi043 ceph-mon[30062]: pool 'c3eefea8-5703-4b79-a4c9-c6adb3b164cb' is full (reached quota's max_objects: 10) 2024-03-21T13:27:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:48 smithi043 ceph-mon[30062]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-21T13:27:49.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:48 smithi043 ceph-mon[30062]: osdmap e832: 8 total, 8 up, 8 in 2024-03-21T13:27:49.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:48 smithi043 ceph-mon[41699]: pool 'c3eefea8-5703-4b79-a4c9-c6adb3b164cb' is full (reached quota's max_objects: 10) 2024-03-21T13:27:49.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:48 smithi043 ceph-mon[41699]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-21T13:27:49.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:48 smithi043 ceph-mon[41699]: osdmap e832: 8 total, 8 up, 8 in 2024-03-21T13:27:50.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:49 smithi155 ceph-mon[33846]: pgmap v1829: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-21T13:27:50.208 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:27:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:27:50.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:49 smithi043 ceph-mon[30062]: pgmap v1829: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-21T13:27:50.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:49 smithi043 ceph-mon[41699]: pgmap v1829: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-21T13:27:51.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:50 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:51.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:50 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:51.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:50 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:27:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:51 smithi155 ceph-mon[33846]: pgmap v1830: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-21T13:27:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:51 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:27:52.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:51 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:27:52.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:51 smithi043 ceph-mon[30062]: pgmap v1830: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-21T13:27:52.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:51 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:27:52.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:51 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:27:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:51 smithi043 ceph-mon[41699]: pgmap v1830: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-21T13:27:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:51 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:27:52.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:51 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:27:54.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:53 smithi155 ceph-mon[33846]: pgmap v1831: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-21T13:27:54.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:53 smithi043 ceph-mon[30062]: pgmap v1831: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-21T13:27:54.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:53 smithi043 ceph-mon[41699]: pgmap v1831: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-21T13:27:56.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:55 smithi155 ceph-mon[33846]: pgmap v1832: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:27:56.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:55 smithi043 ceph-mon[30062]: pgmap v1832: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:27:56.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:55 smithi043 ceph-mon[41699]: pgmap v1832: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:27:58.210 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:57 smithi155 ceph-mon[33846]: pgmap v1833: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:27:58.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:57 smithi043 ceph-mon[30062]: pgmap v1833: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:27:58.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:27:58 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:27:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:27:58.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:57 smithi043 ceph-mon[41699]: pgmap v1833: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:28:00.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:27:59 smithi155 ceph-mon[33846]: pgmap v1834: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 996 B/s rd, 0 op/s 2024-03-21T13:28:00.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:28:00 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:28:00.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:27:59 smithi043 ceph-mon[30062]: pgmap v1834: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 996 B/s rd, 0 op/s 2024-03-21T13:28:00.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:27:59 smithi043 ceph-mon[41699]: pgmap v1834: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 996 B/s rd, 0 op/s 2024-03-21T13:28:01.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:00 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:01.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:00 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:01.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:00 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:02.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:01 smithi155 ceph-mon[33846]: pgmap v1835: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:02.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:01 smithi043 ceph-mon[30062]: pgmap v1835: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:02.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:01 smithi043 ceph-mon[41699]: pgmap v1835: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:04.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:03 smithi155 ceph-mon[33846]: pgmap v1836: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:04.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:03 smithi043 ceph-mon[30062]: pgmap v1836: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:04.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:03 smithi043 ceph-mon[41699]: pgmap v1836: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:05 smithi155 ceph-mon[33846]: pgmap v1837: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:06.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:05 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:28:06.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:05 smithi043 ceph-mon[30062]: pgmap v1837: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:06.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:05 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:28:06.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:05 smithi043 ceph-mon[41699]: pgmap v1837: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:06.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:05 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:28:08.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:07 smithi155 ceph-mon[33846]: pgmap v1838: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:28:08.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:07 smithi043 ceph-mon[30062]: pgmap v1838: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:28:08.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:28:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:28:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:28:08.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:07 smithi043 ceph-mon[41699]: pgmap v1838: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:28:08.732 INFO:tasks.workunit.client.0.smithi043.stderr:+ rados -p dfa6c75f-2043-4860-bf82-1f19a1404959 put threemore /etc/passwd 2024-03-21T13:28:08.772 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota dfa6c75f-2043-4860-bf82-1f19a1404959 max_bytes 0 2024-03-21T13:28:08.864 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.861+0000 7f16c3639640 1 -- 172.21.15.43:0/2925758828 >> v1:172.21.15.43:6789/0 conn(0x7f16bc10a4a0 legacy=0x7f16bc10c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:28:08.864 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.862+0000 7f16c3639640 1 -- 172.21.15.43:0/2925758828 shutdown_connections 2024-03-21T13:28:08.865 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.862+0000 7f16c3639640 1 -- 172.21.15.43:0/2925758828 >> 172.21.15.43:0/2925758828 conn(0x7f16bc0fbea0 msgr2=0x7f16bc0fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:28:08.865 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.862+0000 7f16c3639640 1 -- 172.21.15.43:0/2925758828 shutdown_connections 2024-03-21T13:28:08.865 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.862+0000 7f16c3639640 1 -- 172.21.15.43:0/2925758828 wait complete. 2024-03-21T13:28:08.865 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.862+0000 7f16c3639640 1 Processor -- start 2024-03-21T13:28:08.865 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.862+0000 7f16c3639640 1 -- start start 2024-03-21T13:28:08.865 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.862+0000 7f16c3639640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f16bc1a6550 con 0x7f16bc1026a0 2024-03-21T13:28:08.865 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.862+0000 7f16c3639640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f16bc1a7790 con 0x7f16bc10dfb0 2024-03-21T13:28:08.865 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.862+0000 7f16c3639640 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f16bc1a89d0 con 0x7f16bc10a4a0 2024-03-21T13:28:08.865 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.863+0000 7f16c13d5640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f16bc1026a0 0x7f16bc109ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37168/0 (socket says 172.21.15.43:37168) 2024-03-21T13:28:08.865 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.863+0000 7f16c13d5640 1 -- 172.21.15.43:0/729821419 learned_addr learned my addr 172.21.15.43:0/729821419 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T13:28:08.865 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.863+0000 7f16c0bd4640 1 --1- 172.21.15.43:0/729821419 >> v1:172.21.15.43:6790/0 conn(0x7f16bc10a4a0 0x7f16bc1a13b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:38684/0 (socket says 172.21.15.43:38684) 2024-03-21T13:28:08.865 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.863+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 80545184 0 0) 0x7f16bc1a89d0 con 0x7f16bc10a4a0 2024-03-21T13:28:08.865 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.863+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16a8003520 con 0x7f16bc10a4a0 2024-03-21T13:28:08.866 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.863+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 661908716 0 0) 0x7f16bc1a6550 con 0x7f16bc1026a0 2024-03-21T13:28:08.866 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.863+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16bc1a89d0 con 0x7f16bc1026a0 2024-03-21T13:28:08.866 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.863+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3933775322 0 0) 0x7f16bc1a7790 con 0x7f16bc10dfb0 2024-03-21T13:28:08.866 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.863+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16bc1a6550 con 0x7f16bc10dfb0 2024-03-21T13:28:08.866 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.863+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1577958730 0 0) 0x7f16a8003520 con 0x7f16bc10a4a0 2024-03-21T13:28:08.866 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.863+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f16bc1a7790 con 0x7f16bc10a4a0 2024-03-21T13:28:08.866 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.863+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f16ac0044f0 con 0x7f16bc10a4a0 2024-03-21T13:28:08.866 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.863+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1223565492 0 0) 0x7f16bc1a89d0 con 0x7f16bc1026a0 2024-03-21T13:28:08.866 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.863+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f16a8003520 con 0x7f16bc1026a0 2024-03-21T13:28:08.866 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.863+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f16b8001890 con 0x7f16bc1026a0 2024-03-21T13:28:08.866 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.863+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2871433649 0 0) 0x7f16bc1a6550 con 0x7f16bc10dfb0 2024-03-21T13:28:08.866 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.863+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f16bc1a89d0 con 0x7f16bc10dfb0 2024-03-21T13:28:08.866 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.863+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f16b4001af0 con 0x7f16bc10dfb0 2024-03-21T13:28:08.867 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.863+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 <== mon.2 v1:172.21.15.43:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2748962986 0 0) 0x7f16bc1a7790 con 0x7f16bc10a4a0 2024-03-21T13:28:08.867 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.863+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 >> v1:172.21.15.155:6789/0 conn(0x7f16bc10dfb0 legacy=0x7f16bc1a4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:28:08.867 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.863+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 >> v1:172.21.15.43:6789/0 conn(0x7f16bc1026a0 legacy=0x7f16bc109ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:28:08.867 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.863+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 --> v1:172.21.15.43:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16bc1a9c10 con 0x7f16bc10a4a0 2024-03-21T13:28:08.867 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.863+0000 7f16c3639640 1 -- 172.21.15.43:0/729821419 --> v1:172.21.15.43:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16bc1a6780 con 0x7f16bc10a4a0 2024-03-21T13:28:08.867 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.863+0000 7f16c3639640 1 -- 172.21.15.43:0/729821419 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f16bc1a6c50 con 0x7f16bc10a4a0 2024-03-21T13:28:08.867 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.864+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 <== mon.2 v1:172.21.15.43:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f16ac001b30 con 0x7f16bc10a4a0 2024-03-21T13:28:08.867 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.864+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 <== mon.2 v1:172.21.15.43:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f16ac004c30 con 0x7f16bc10a4a0 2024-03-21T13:28:08.867 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.864+0000 7f16c3639640 1 -- 172.21.15.43:0/729821419 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a0005080 con 0x7f16bc10a4a0 2024-03-21T13:28:08.867 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.865+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 <== mon.2 v1:172.21.15.43:6790/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7f16ac004eb0 con 0x7f16bc10a4a0 2024-03-21T13:28:08.868 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.865+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 <== mon.2 v1:172.21.15.43:6790/0 8 ==== osd_map(832..832 src has 251..832) v4 ==== 8186+0+0 (unknown 2089832010 0 0) 0x7f16ac089f40 con 0x7f16bc10a4a0 2024-03-21T13:28:08.868 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.865+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=833}) v3 -- 0x7f16bc1a7790 con 0x7f16bc10a4a0 2024-03-21T13:28:08.869 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:08.870+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 <== mon.2 v1:172.21.15.43:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f16ac056d90 con 0x7f16bc10a4a0 2024-03-21T13:28:09.055 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:09.055+0000 7f16c3639640 1 -- 172.21.15.43:0/729821419 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f16a0005370 con 0x7f16bc10a4a0 2024-03-21T13:28:09.895 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:09.895+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 <== mon.2 v1:172.21.15.43:6790/0 10 ==== osd_map(833..833 src has 251..833) v4 ==== 628+0+0 (unknown 692183385 0 0) 0x7f16ac004c30 con 0x7f16bc10a4a0 2024-03-21T13:28:09.896 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:09.895+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=834}) v3 -- 0x7f16bc1a6550 con 0x7f16bc10a4a0 2024-03-21T13:28:09.900 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:09.901+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 <== mon.2 v1:172.21.15.43:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool dfa6c75f-2043-4860-bf82-1f19a1404959 v833) v1 ==== 217+0+0 (unknown 4151982688 0 0) 0x7f16ac05a7e0 con 0x7f16bc10a4a0 2024-03-21T13:28:10.019 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:10.019+0000 7f16c3639640 1 -- 172.21.15.43:0/729821419 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f16a0005b80 con 0x7f16bc10a4a0 2024-03-21T13:28:10.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:28:10 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:28:10.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:09 smithi155 ceph-mon[33846]: pgmap v1839: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:10.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:09 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/729821419' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T13:28:10.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:09 smithi155 ceph-mon[33846]: from='client.58979 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T13:28:10.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:09 smithi043 ceph-mon[30062]: pgmap v1839: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:10.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:09 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/729821419' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T13:28:10.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:09 smithi043 ceph-mon[30062]: from='client.58979 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T13:28:10.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:09 smithi043 ceph-mon[41699]: pgmap v1839: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:10.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:09 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/729821419' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T13:28:10.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:09 smithi043 ceph-mon[41699]: from='client.58979 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T13:28:10.913 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:10.913+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 <== mon.2 v1:172.21.15.43:6790/0 12 ==== osd_map(834..834 src has 251..834) v4 ==== 628+0+0 (unknown 696303483 0 0) 0x7f16ac087b70 con 0x7f16bc10a4a0 2024-03-21T13:28:10.913 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:10.913+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=835}) v3 -- 0x7f16a807ca90 con 0x7f16bc10a4a0 2024-03-21T13:28:10.917 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:10.917+0000 7f16b27fc640 1 -- 172.21.15.43:0/729821419 <== mon.2 v1:172.21.15.43:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool dfa6c75f-2043-4860-bf82-1f19a1404959 v834) v1 ==== 217+0+0 (unknown 2172345027 0 0) 0x7f16ac05f0a0 con 0x7f16bc10a4a0 2024-03-21T13:28:10.917 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_bytes = 0 for pool dfa6c75f-2043-4860-bf82-1f19a1404959 2024-03-21T13:28:10.921 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:10.921+0000 7f16c3639640 1 -- 172.21.15.43:0/729821419 >> v1:172.21.15.43:6800/4199387216 conn(0x7f16a806f0e0 legacy=0x7f16a80715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:28:10.921 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:10.921+0000 7f16c3639640 1 -- 172.21.15.43:0/729821419 >> v1:172.21.15.43:6790/0 conn(0x7f16bc10a4a0 legacy=0x7f16bc1a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:28:10.921 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:10.921+0000 7f16c3639640 1 -- 172.21.15.43:0/729821419 shutdown_connections 2024-03-21T13:28:10.921 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:10.921+0000 7f16c3639640 1 -- 172.21.15.43:0/729821419 >> 172.21.15.43:0/729821419 conn(0x7f16bc0fbea0 msgr2=0x7f16bc0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:28:10.921 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:10.921+0000 7f16c3639640 1 -- 172.21.15.43:0/729821419 shutdown_connections 2024-03-21T13:28:10.921 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:10.921+0000 7f16c3639640 1 -- 172.21.15.43:0/729821419 wait complete. 2024-03-21T13:28:10.932 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool set-quota dfa6c75f-2043-4860-bf82-1f19a1404959 max_objects 0 2024-03-21T13:28:11.021 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.020+0000 7fd27cdfe640 1 -- 172.21.15.43:0/1243463205 >> v1:172.21.15.43:6790/0 conn(0x7fd278102700 legacy=0x7fd278102ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:28:11.021 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.021+0000 7fd27cdfe640 1 -- 172.21.15.43:0/1243463205 shutdown_connections 2024-03-21T13:28:11.021 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.021+0000 7fd27cdfe640 1 -- 172.21.15.43:0/1243463205 >> 172.21.15.43:0/1243463205 conn(0x7fd2780fbea0 msgr2=0x7fd2780fe2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:28:11.021 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.021+0000 7fd27cdfe640 1 -- 172.21.15.43:0/1243463205 shutdown_connections 2024-03-21T13:28:11.021 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.021+0000 7fd27cdfe640 1 -- 172.21.15.43:0/1243463205 wait complete. 2024-03-21T13:28:11.021 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.021+0000 7fd27cdfe640 1 Processor -- start 2024-03-21T13:28:11.024 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.022+0000 7fd27cdfe640 1 -- start start 2024-03-21T13:28:11.024 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.022+0000 7fd27cdfe640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd2781a6610 con 0x7fd27810e010 2024-03-21T13:28:11.024 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.022+0000 7fd27cdfe640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd2781a7810 con 0x7fd27810a500 2024-03-21T13:28:11.024 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.022+0000 7fd27cdfe640 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fd2781a8a50 con 0x7fd278102700 2024-03-21T13:28:11.024 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.022+0000 7fd27659c640 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7fd278102700 0x7fd278109db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:38708/0 (socket says 172.21.15.43:38708) 2024-03-21T13:28:11.024 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.022+0000 7fd27659c640 1 -- 172.21.15.43:0/4023669333 learned_addr learned my addr 172.21.15.43:0/4023669333 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T13:28:11.024 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.022+0000 7fd276d9d640 1 --1- 172.21.15.43:0/4023669333 >> v1:172.21.15.43:6789/0 conn(0x7fd27810e010 0x7fd2781a4d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37196/0 (socket says 172.21.15.43:37196) 2024-03-21T13:28:11.024 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.022+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3532865973 0 0) 0x7fd2781a6610 con 0x7fd27810e010 2024-03-21T13:28:11.024 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.022+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd264003160 con 0x7fd27810e010 2024-03-21T13:28:11.024 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.022+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2705972517 0 0) 0x7fd2781a8a50 con 0x7fd278102700 2024-03-21T13:28:11.024 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.022+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2781a6610 con 0x7fd278102700 2024-03-21T13:28:11.024 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.022+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3269860488 0 0) 0x7fd2781a7810 con 0x7fd27810a500 2024-03-21T13:28:11.024 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.022+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2781a8a50 con 0x7fd27810a500 2024-03-21T13:28:11.025 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.022+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1528721877 0 0) 0x7fd264003160 con 0x7fd27810e010 2024-03-21T13:28:11.025 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.022+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd2781a7810 con 0x7fd27810e010 2024-03-21T13:28:11.025 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.022+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1504573467 0 0) 0x7fd2781a6610 con 0x7fd278102700 2024-03-21T13:28:11.025 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.022+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fd264003160 con 0x7fd278102700 2024-03-21T13:28:11.025 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.022+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fd270001750 con 0x7fd27810e010 2024-03-21T13:28:11.025 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.022+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fd2680041f0 con 0x7fd278102700 2024-03-21T13:28:11.025 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.023+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 <== mon.2 v1:172.21.15.43:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3529997841 0 0) 0x7fd264003160 con 0x7fd278102700 2024-03-21T13:28:11.025 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.023+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 >> v1:172.21.15.155:6789/0 conn(0x7fd27810a500 legacy=0x7fd2781a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:28:11.025 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.023+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 >> v1:172.21.15.43:6789/0 conn(0x7fd27810e010 legacy=0x7fd2781a4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:28:11.025 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.023+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 --> v1:172.21.15.43:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2781a9c90 con 0x7fd278102700 2024-03-21T13:28:11.025 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.023+0000 7fd27cdfe640 1 -- 172.21.15.43:0/4023669333 --> v1:172.21.15.43:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2781a6840 con 0x7fd278102700 2024-03-21T13:28:11.025 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.023+0000 7fd27cdfe640 1 -- 172.21.15.43:0/4023669333 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2781a6d10 con 0x7fd278102700 2024-03-21T13:28:11.025 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.023+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 <== mon.2 v1:172.21.15.43:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7fd2680010c0 con 0x7fd278102700 2024-03-21T13:28:11.025 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.023+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 <== mon.2 v1:172.21.15.43:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7fd268004bb0 con 0x7fd278102700 2024-03-21T13:28:11.025 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.024+0000 7fd27cdfe640 1 -- 172.21.15.43:0/4023669333 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd258005080 con 0x7fd278102700 2024-03-21T13:28:11.026 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.024+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 <== mon.2 v1:172.21.15.43:6790/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7fd268003810 con 0x7fd278102700 2024-03-21T13:28:11.026 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.025+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 <== mon.2 v1:172.21.15.43:6790/0 8 ==== osd_map(834..834 src has 251..834) v4 ==== 8186+0+0 (unknown 1103407224 0 0) 0x7fd268089df0 con 0x7fd278102700 2024-03-21T13:28:11.026 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.025+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=835}) v3 -- 0x7fd264003160 con 0x7fd278102700 2024-03-21T13:28:11.029 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.029+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 <== mon.2 v1:172.21.15.43:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7fd2680044d0 con 0x7fd278102700 2024-03-21T13:28:11.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:10 smithi155 ceph-mon[33846]: from='client.58979 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]': finished 2024-03-21T13:28:11.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:10 smithi155 ceph-mon[33846]: osdmap e833: 8 total, 8 up, 8 in 2024-03-21T13:28:11.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:10 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:11.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:10 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/729821419' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T13:28:11.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:10 smithi155 ceph-mon[33846]: from='client.58979 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T13:28:11.212 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.212+0000 7fd27cdfe640 1 -- 172.21.15.43:0/4023669333 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fd258005370 con 0x7fd278102700 2024-03-21T13:28:11.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:10 smithi043 ceph-mon[30062]: from='client.58979 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]': finished 2024-03-21T13:28:11.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:10 smithi043 ceph-mon[30062]: osdmap e833: 8 total, 8 up, 8 in 2024-03-21T13:28:11.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:10 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:11.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:10 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/729821419' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T13:28:11.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:10 smithi043 ceph-mon[30062]: from='client.58979 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T13:28:11.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:10 smithi043 ceph-mon[41699]: from='client.58979 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]': finished 2024-03-21T13:28:11.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:10 smithi043 ceph-mon[41699]: osdmap e833: 8 total, 8 up, 8 in 2024-03-21T13:28:11.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:10 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:11.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:10 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/729821419' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T13:28:11.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:10 smithi043 ceph-mon[41699]: from='client.58979 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-21T13:28:11.921 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.921+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 <== mon.2 v1:172.21.15.43:6790/0 10 ==== osd_map(835..835 src has 251..835) v4 ==== 628+0+0 (unknown 3561911555 0 0) 0x7fd268087a20 con 0x7fd278102700 2024-03-21T13:28:11.921 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.921+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=836}) v3 -- 0x7fd2781a7810 con 0x7fd278102700 2024-03-21T13:28:11.926 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:11.927+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 <== mon.2 v1:172.21.15.43:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool dfa6c75f-2043-4860-bf82-1f19a1404959 v835) v1 ==== 221+0+0 (unknown 106897747 0 0) 0x7fd268056c40 con 0x7fd278102700 2024-03-21T13:28:12.039 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:12.039+0000 7fd27cdfe640 1 -- 172.21.15.43:0/4023669333 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fd258001150 con 0x7fd278102700 2024-03-21T13:28:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:11 smithi155 ceph-mon[33846]: pgmap v1841: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:28:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:11 smithi155 ceph-mon[33846]: from='client.58979 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]': finished 2024-03-21T13:28:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:11 smithi155 ceph-mon[33846]: osdmap e834: 8 total, 8 up, 8 in 2024-03-21T13:28:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:11 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4023669333' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T13:28:12.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:11 smithi155 ceph-mon[33846]: from='client.58985 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T13:28:12.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:11 smithi043 ceph-mon[30062]: pgmap v1841: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:28:12.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:11 smithi043 ceph-mon[30062]: from='client.58979 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]': finished 2024-03-21T13:28:12.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:11 smithi043 ceph-mon[30062]: osdmap e834: 8 total, 8 up, 8 in 2024-03-21T13:28:12.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:11 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4023669333' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T13:28:12.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:11 smithi043 ceph-mon[30062]: from='client.58985 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T13:28:12.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:11 smithi043 ceph-mon[41699]: pgmap v1841: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:28:12.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:11 smithi043 ceph-mon[41699]: from='client.58979 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_bytes", "val": "0"}]': finished 2024-03-21T13:28:12.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:11 smithi043 ceph-mon[41699]: osdmap e834: 8 total, 8 up, 8 in 2024-03-21T13:28:12.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:11 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4023669333' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T13:28:12.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:11 smithi043 ceph-mon[41699]: from='client.58985 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T13:28:12.926 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:12.926+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 <== mon.2 v1:172.21.15.43:6790/0 12 ==== osd_map(836..836 src has 251..836) v4 ==== 628+0+0 (unknown 1116636153 0 0) 0x7fd268052580 con 0x7fd278102700 2024-03-21T13:28:12.926 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:12.926+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=837}) v3 -- 0x7fd2781a8a50 con 0x7fd278102700 2024-03-21T13:28:12.933 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:12.933+0000 7fd26ffff640 1 -- 172.21.15.43:0/4023669333 <== mon.2 v1:172.21.15.43:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool dfa6c75f-2043-4860-bf82-1f19a1404959 v836) v1 ==== 221+0+0 (unknown 2312736743 0 0) 0x7fd26805a690 con 0x7fd278102700 2024-03-21T13:28:12.933 INFO:tasks.workunit.client.0.smithi043.stderr:set-quota max_objects = 0 for pool dfa6c75f-2043-4860-bf82-1f19a1404959 2024-03-21T13:28:12.936 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:12.936+0000 7fd27cdfe640 1 -- 172.21.15.43:0/4023669333 >> v1:172.21.15.43:6800/4199387216 conn(0x7fd26406eeb0 legacy=0x7fd264071370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:28:12.936 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:12.936+0000 7fd27cdfe640 1 -- 172.21.15.43:0/4023669333 >> v1:172.21.15.43:6790/0 conn(0x7fd278102700 legacy=0x7fd278109db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:28:12.936 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:12.936+0000 7fd27cdfe640 1 -- 172.21.15.43:0/4023669333 shutdown_connections 2024-03-21T13:28:12.936 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:12.936+0000 7fd27cdfe640 1 -- 172.21.15.43:0/4023669333 >> 172.21.15.43:0/4023669333 conn(0x7fd2780fbea0 msgr2=0x7fd2780fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:28:12.936 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:12.936+0000 7fd27cdfe640 1 -- 172.21.15.43:0/4023669333 shutdown_connections 2024-03-21T13:28:12.936 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:12.936+0000 7fd27cdfe640 1 -- 172.21.15.43:0/4023669333 wait complete. 2024-03-21T13:28:12.947 INFO:tasks.workunit.client.0.smithi043.stderr:+ sleep 30 2024-03-21T13:28:13.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:12 smithi155 ceph-mon[33846]: from='client.58985 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]': finished 2024-03-21T13:28:13.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:12 smithi155 ceph-mon[33846]: osdmap e835: 8 total, 8 up, 8 in 2024-03-21T13:28:13.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:12 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/4023669333' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T13:28:13.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:12 smithi155 ceph-mon[33846]: from='client.58985 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T13:28:13.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:12 smithi043 ceph-mon[30062]: from='client.58985 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]': finished 2024-03-21T13:28:13.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:12 smithi043 ceph-mon[30062]: osdmap e835: 8 total, 8 up, 8 in 2024-03-21T13:28:13.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:12 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/4023669333' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T13:28:13.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:12 smithi043 ceph-mon[30062]: from='client.58985 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T13:28:13.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:12 smithi043 ceph-mon[41699]: from='client.58985 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]': finished 2024-03-21T13:28:13.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:12 smithi043 ceph-mon[41699]: osdmap e835: 8 total, 8 up, 8 in 2024-03-21T13:28:13.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:12 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/4023669333' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T13:28:13.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:12 smithi043 ceph-mon[41699]: from='client.58985 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]: dispatch 2024-03-21T13:28:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:13 smithi155 ceph-mon[33846]: pgmap v1844: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:13 smithi155 ceph-mon[33846]: from='client.58985 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]': finished 2024-03-21T13:28:14.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:13 smithi155 ceph-mon[33846]: osdmap e836: 8 total, 8 up, 8 in 2024-03-21T13:28:14.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:13 smithi043 ceph-mon[30062]: pgmap v1844: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:14.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:13 smithi043 ceph-mon[30062]: from='client.58985 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]': finished 2024-03-21T13:28:14.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:13 smithi043 ceph-mon[30062]: osdmap e836: 8 total, 8 up, 8 in 2024-03-21T13:28:14.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:13 smithi043 ceph-mon[41699]: pgmap v1844: 188 pgs: 188 active+clean; 859 KiB data, 882 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:14.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:13 smithi043 ceph-mon[41699]: from='client.58985 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "field": "max_objects", "val": "0"}]': finished 2024-03-21T13:28:14.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:13 smithi043 ceph-mon[41699]: osdmap e836: 8 total, 8 up, 8 in 2024-03-21T13:28:16.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:15 smithi155 ceph-mon[33846]: pgmap v1846: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.5 KiB/s wr, 1 op/s 2024-03-21T13:28:16.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:15 smithi043 ceph-mon[30062]: pgmap v1846: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.5 KiB/s wr, 1 op/s 2024-03-21T13:28:16.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:15 smithi043 ceph-mon[41699]: pgmap v1846: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.5 KiB/s wr, 1 op/s 2024-03-21T13:28:18.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:17 smithi155 ceph-mon[33846]: pgmap v1847: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-03-21T13:28:18.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:17 smithi043 ceph-mon[30062]: pgmap v1847: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-03-21T13:28:18.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:28:18 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:28:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:28:18.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:17 smithi043 ceph-mon[41699]: pgmap v1847: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-03-21T13:28:20.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:19 smithi155 ceph-mon[33846]: pgmap v1848: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-03-21T13:28:20.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:28:20 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:28:20.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:19 smithi043 ceph-mon[30062]: pgmap v1848: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-03-21T13:28:20.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:19 smithi043 ceph-mon[41699]: pgmap v1848: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 1 op/s 2024-03-21T13:28:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:20 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:21.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:20 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:28:21.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:20 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:21.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:20 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:28:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:20 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:21.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:20 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:28:22.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:21 smithi155 ceph-mon[33846]: pgmap v1849: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.4 KiB/s wr, 1 op/s 2024-03-21T13:28:22.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:21 smithi043 ceph-mon[30062]: pgmap v1849: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.4 KiB/s wr, 1 op/s 2024-03-21T13:28:22.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:21 smithi043 ceph-mon[41699]: pgmap v1849: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.4 KiB/s wr, 1 op/s 2024-03-21T13:28:24.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:23 smithi155 ceph-mon[33846]: pgmap v1850: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-03-21T13:28:24.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:23 smithi043 ceph-mon[30062]: pgmap v1850: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-03-21T13:28:24.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:23 smithi043 ceph-mon[41699]: pgmap v1850: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 1.2 KiB/s wr, 1 op/s 2024-03-21T13:28:26.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:25 smithi155 ceph-mon[33846]: pgmap v1851: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 KiB/s wr, 1 op/s 2024-03-21T13:28:26.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:25 smithi043 ceph-mon[30062]: pgmap v1851: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 KiB/s wr, 1 op/s 2024-03-21T13:28:26.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:25 smithi043 ceph-mon[41699]: pgmap v1851: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 KiB/s wr, 1 op/s 2024-03-21T13:28:28.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:27 smithi155 ceph-mon[33846]: pgmap v1852: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:28:28.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:27 smithi043 ceph-mon[30062]: pgmap v1852: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:28:28.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:28:28 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:28:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:28:28.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:27 smithi043 ceph-mon[41699]: pgmap v1852: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:28:30.207 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:29 smithi155 ceph-mon[33846]: pgmap v1853: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:30.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:28:30 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:28:30.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:29 smithi043 ceph-mon[30062]: pgmap v1853: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:30.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:29 smithi043 ceph-mon[41699]: pgmap v1853: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:31.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:30 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:31.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:30 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:31.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:30 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:32.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:31 smithi043 ceph-mon[30062]: pgmap v1854: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:28:32.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:31 smithi043 ceph-mon[41699]: pgmap v1854: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:28:32.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:31 smithi155 ceph-mon[33846]: pgmap v1854: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-21T13:28:34.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:33 smithi043 ceph-mon[30062]: pgmap v1855: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:34.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:33 smithi043 ceph-mon[41699]: pgmap v1855: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:34.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:33 smithi155 ceph-mon[33846]: pgmap v1855: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:36.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:35 smithi043 ceph-mon[30062]: pgmap v1856: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:36.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:35 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:28:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:35 smithi043 ceph-mon[41699]: pgmap v1856: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:36.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:35 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:28:36.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:35 smithi155 ceph-mon[33846]: pgmap v1856: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:36.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:35 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-21T13:28:38.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:37 smithi043 ceph-mon[30062]: pgmap v1857: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:38.234 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:28:38 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:28:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:28:38.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:37 smithi043 ceph-mon[41699]: pgmap v1857: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:38.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:37 smithi155 ceph-mon[33846]: pgmap v1857: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:40.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:39 smithi043 ceph-mon[30062]: pgmap v1858: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:40.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:39 smithi043 ceph-mon[41699]: pgmap v1858: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:40.259 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:39 smithi155 ceph-mon[33846]: pgmap v1858: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-21T13:28:40.259 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:28:40 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:28:42.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:41 smithi043 ceph-mon[30062]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:42.234 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:41 smithi043 ceph-mon[30062]: pgmap v1859: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:41 smithi043 ceph-mon[41699]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:42.234 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:41 smithi043 ceph-mon[41699]: pgmap v1859: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:42.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:41 smithi155 ceph-mon[33846]: from='client.24457 v1:172.21.15.155:0/2955358153' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-21T13:28:42.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:41 smithi155 ceph-mon[33846]: pgmap v1859: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:42.948 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool delete dfa6c75f-2043-4860-bf82-1f19a1404959 dfa6c75f-2043-4860-bf82-1f19a1404959 --yes-i-really-really-mean-it 2024-03-21T13:28:43.033 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.032+0000 7f16ea017640 1 -- 172.21.15.43:0/3450628150 >> v1:172.21.15.43:6789/0 conn(0x7f16e410df60 legacy=0x7f16e4110400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:28:43.033 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.032+0000 7f16ea017640 1 -- 172.21.15.43:0/3450628150 shutdown_connections 2024-03-21T13:28:43.033 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.032+0000 7f16ea017640 1 -- 172.21.15.43:0/3450628150 >> 172.21.15.43:0/3450628150 conn(0x7f16e4075700 msgr2=0x7f16e4075b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:28:43.033 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.032+0000 7f16ea017640 1 -- 172.21.15.43:0/3450628150 shutdown_connections 2024-03-21T13:28:43.033 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.032+0000 7f16ea017640 1 -- 172.21.15.43:0/3450628150 wait complete. 2024-03-21T13:28:43.033 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.033+0000 7f16ea017640 1 Processor -- start 2024-03-21T13:28:43.036 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.033+0000 7f16ea017640 1 -- start start 2024-03-21T13:28:43.036 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.033+0000 7f16ea017640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f16e41a6510 con 0x7f16e40fff80 2024-03-21T13:28:43.036 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.033+0000 7f16ea017640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f16e41a7750 con 0x7f16e410a3f0 2024-03-21T13:28:43.036 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.033+0000 7f16ea017640 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f16e41a8990 con 0x7f16e410df60 2024-03-21T13:28:43.036 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.033+0000 7f16e37fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f16e40fff80 0x7f16e4106ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41182/0 (socket says 172.21.15.43:41182) 2024-03-21T13:28:43.036 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.033+0000 7f16e3fff640 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7f16e410df60 0x7f16e41a4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:60930/0 (socket says 172.21.15.43:60930) 2024-03-21T13:28:43.036 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.033+0000 7f16e37fe640 1 -- 172.21.15.43:0/3794097207 learned_addr learned my addr 172.21.15.43:0/3794097207 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T13:28:43.036 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.033+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2970025099 0 0) 0x7f16e41a8990 con 0x7f16e410df60 2024-03-21T13:28:43.037 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.033+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16cc003520 con 0x7f16e410df60 2024-03-21T13:28:43.037 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.033+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 899595786 0 0) 0x7f16e41a6510 con 0x7f16e40fff80 2024-03-21T13:28:43.037 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.033+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16e41a8990 con 0x7f16e40fff80 2024-03-21T13:28:43.037 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.033+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 198192338 0 0) 0x7f16e41a7750 con 0x7f16e410a3f0 2024-03-21T13:28:43.037 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.033+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16e41a6510 con 0x7f16e410a3f0 2024-03-21T13:28:43.037 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.033+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4126716987 0 0) 0x7f16cc003520 con 0x7f16e410df60 2024-03-21T13:28:43.037 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.034+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f16e41a7750 con 0x7f16e410df60 2024-03-21T13:28:43.037 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.034+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f16dc004750 con 0x7f16e410df60 2024-03-21T13:28:43.037 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.034+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2045312690 0 0) 0x7f16e41a8990 con 0x7f16e40fff80 2024-03-21T13:28:43.037 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.034+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f16cc003520 con 0x7f16e40fff80 2024-03-21T13:28:43.037 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.034+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f16d4001200 con 0x7f16e40fff80 2024-03-21T13:28:43.037 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.034+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1949996574 0 0) 0x7f16e41a6510 con 0x7f16e410a3f0 2024-03-21T13:28:43.038 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.034+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f16e41a8990 con 0x7f16e410a3f0 2024-03-21T13:28:43.038 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.034+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f16d80017f0 con 0x7f16e410a3f0 2024-03-21T13:28:43.038 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.034+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 <== mon.0 v1:172.21.15.43:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3645859798 0 0) 0x7f16cc003520 con 0x7f16e40fff80 2024-03-21T13:28:43.038 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.034+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 >> v1:172.21.15.43:6790/0 conn(0x7f16e410df60 legacy=0x7f16e41a4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:28:43.038 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.034+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 >> v1:172.21.15.155:6789/0 conn(0x7f16e410a3f0 legacy=0x7f16e41a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:28:43.038 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.034+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e41a9bd0 con 0x7f16e40fff80 2024-03-21T13:28:43.038 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.034+0000 7f16ea017640 1 -- 172.21.15.43:0/3794097207 --> v1:172.21.15.43:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e41a6740 con 0x7f16e40fff80 2024-03-21T13:28:43.038 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.034+0000 7f16ea017640 1 -- 172.21.15.43:0/3794097207 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e41a6c90 con 0x7f16e40fff80 2024-03-21T13:28:43.038 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.034+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 <== mon.0 v1:172.21.15.43:6789/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f16d4003a80 con 0x7f16e40fff80 2024-03-21T13:28:43.038 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.034+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 <== mon.0 v1:172.21.15.43:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f16d4005b00 con 0x7f16e40fff80 2024-03-21T13:28:43.038 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.035+0000 7f16ea017640 1 -- 172.21.15.43:0/3794097207 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d00050b0 con 0x7f16e40fff80 2024-03-21T13:28:43.038 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.035+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 <== mon.0 v1:172.21.15.43:6789/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7f16d40035f0 con 0x7f16e40fff80 2024-03-21T13:28:43.039 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.036+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 <== mon.0 v1:172.21.15.43:6789/0 8 ==== osd_map(836..836 src has 251..836) v4 ==== 8186+0+0 (unknown 104432174 0 0) 0x7f16d4089f00 con 0x7f16e40fff80 2024-03-21T13:28:43.039 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.036+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=837}) v3 -- 0x7f16cc003520 con 0x7f16e40fff80 2024-03-21T13:28:43.041 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.041+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 <== mon.0 v1:172.21.15.43:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f16d4056d50 con 0x7f16e40fff80 2024-03-21T13:28:43.224 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.224+0000 7f16ea017640 1 -- 172.21.15.43:0/3794097207 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pool2": "dfa6c75f-2043-4860-bf82-1f19a1404959", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f16d00053a0 con 0x7f16e40fff80 2024-03-21T13:28:43.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:42 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:28:43.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:42 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:28:43.233 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:42 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:28:43.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:42 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:28:43.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:42 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:28:43.233 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:42 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:28:43.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:42 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T13:28:43.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:42 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:28:43.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:42 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:28:43.980 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.978+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 <== mon.0 v1:172.21.15.43:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pool2": "dfa6c75f-2043-4860-bf82-1f19a1404959", "yes_i_really_really_mean_it": true}]=0 pool 'dfa6c75f-2043-4860-bf82-1f19a1404959' removed v837) v1 ==== 248+0+0 (unknown 2028751657 0 0) 0x7f16d405a7a0 con 0x7f16e40fff80 2024-03-21T13:28:43.980 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.979+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 <== mon.0 v1:172.21.15.43:6789/0 11 ==== osd_map(837..837 src has 251..837) v4 ==== 296+0+0 (unknown 2765917469 0 0) 0x7f16d4087b30 con 0x7f16e40fff80 2024-03-21T13:28:43.981 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:43.979+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 --> v1:172.21.15.43:6789/0 -- mon_subscribe({osdmap=838}) v3 -- 0x7f16e41a8990 con 0x7f16e40fff80 2024-03-21T13:28:44.098 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.097+0000 7f16ea017640 1 -- 172.21.15.43:0/3794097207 --> v1:172.21.15.43:6789/0 -- mon_command({"prefix": "osd pool delete", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pool2": "dfa6c75f-2043-4860-bf82-1f19a1404959", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f16d0005dd0 con 0x7f16e40fff80 2024-03-21T13:28:44.098 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.097+0000 7f16e0ff9640 1 -- 172.21.15.43:0/3794097207 <== mon.0 v1:172.21.15.43:6789/0 12 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pool2": "dfa6c75f-2043-4860-bf82-1f19a1404959", "yes_i_really_really_mean_it": true}]=0 pool 'dfa6c75f-2043-4860-bf82-1f19a1404959' does not exist v837) v1 ==== 255+0+0 (unknown 2102801555 0 0) 0x7f16d405f060 con 0x7f16e40fff80 2024-03-21T13:28:44.098 INFO:tasks.workunit.client.0.smithi043.stderr:pool 'dfa6c75f-2043-4860-bf82-1f19a1404959' does not exist 2024-03-21T13:28:44.100 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.099+0000 7f16ea017640 1 -- 172.21.15.43:0/3794097207 >> v1:172.21.15.43:6800/4199387216 conn(0x7f16cc06f0a0 legacy=0x7f16cc071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:28:44.100 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.099+0000 7f16ea017640 1 -- 172.21.15.43:0/3794097207 >> v1:172.21.15.43:6789/0 conn(0x7f16e40fff80 legacy=0x7f16e4106ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:28:44.101 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.100+0000 7f16ea017640 1 -- 172.21.15.43:0/3794097207 shutdown_connections 2024-03-21T13:28:44.101 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.100+0000 7f16ea017640 1 -- 172.21.15.43:0/3794097207 >> 172.21.15.43:0/3794097207 conn(0x7f16e4075700 msgr2=0x7f16e4067ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:28:44.101 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.100+0000 7f16ea017640 1 -- 172.21.15.43:0/3794097207 shutdown_connections 2024-03-21T13:28:44.101 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.100+0000 7f16ea017640 1 -- 172.21.15.43:0/3794097207 wait complete. 2024-03-21T13:28:44.109 INFO:tasks.workunit.client.0.smithi043.stderr:+ ceph osd pool delete c3eefea8-5703-4b79-a4c9-c6adb3b164cb c3eefea8-5703-4b79-a4c9-c6adb3b164cb --yes-i-really-really-mean-it 2024-03-21T13:28:44.199 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.193+0000 7f0c93325640 1 -- 172.21.15.43:0/137613495 >> v1:172.21.15.43:6790/0 conn(0x7f0c8c10a370 legacy=0x7f0c8c10c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:28:44.199 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.194+0000 7f0c93325640 1 -- 172.21.15.43:0/137613495 shutdown_connections 2024-03-21T13:28:44.199 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.194+0000 7f0c93325640 1 -- 172.21.15.43:0/137613495 >> 172.21.15.43:0/137613495 conn(0x7f0c8c0fcf80 msgr2=0x7f0c8c0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:28:44.200 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.194+0000 7f0c93325640 1 -- 172.21.15.43:0/137613495 shutdown_connections 2024-03-21T13:28:44.200 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.194+0000 7f0c93325640 1 -- 172.21.15.43:0/137613495 wait complete. 2024-03-21T13:28:44.200 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.194+0000 7f0c93325640 1 Processor -- start 2024-03-21T13:28:44.200 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.194+0000 7f0c93325640 1 -- start start 2024-03-21T13:28:44.200 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.194+0000 7f0c93325640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0c8c1a6570 con 0x7f0c8c10df70 2024-03-21T13:28:44.200 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.194+0000 7f0c93325640 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0c8c1a77b0 con 0x7f0c8c10a370 2024-03-21T13:28:44.201 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.194+0000 7f0c93325640 1 -- --> v1:172.21.15.43:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0c8c1a89f0 con 0x7f0c8c1075d0 2024-03-21T13:28:44.201 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.194+0000 7f0c910c1640 1 --1- >> v1:172.21.15.43:6790/0 conn(0x7f0c8c1075d0 0x7f0c8c106920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.43:6790/0 says I am v1:172.21.15.43:60962/0 (socket says 172.21.15.43:60962) 2024-03-21T13:28:44.201 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.194+0000 7f0c910c1640 1 -- 172.21.15.43:0/869710287 learned_addr learned my addr 172.21.15.43:0/869710287 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-03-21T13:28:44.201 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.195+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 <== mon.2 v1:172.21.15.43:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2900898974 0 0) 0x7f0c8c1a89f0 con 0x7f0c8c1075d0 2024-03-21T13:28:44.201 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.195+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 --> v1:172.21.15.43:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c78003520 con 0x7f0c8c1075d0 2024-03-21T13:28:44.201 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.195+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2901509007 0 0) 0x7f0c8c1a6570 con 0x7f0c8c10df70 2024-03-21T13:28:44.202 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.195+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c8c1a89f0 con 0x7f0c8c10df70 2024-03-21T13:28:44.202 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.195+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1475653745 0 0) 0x7f0c8c1a77b0 con 0x7f0c8c10a370 2024-03-21T13:28:44.202 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.195+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c8c1a6570 con 0x7f0c8c10a370 2024-03-21T13:28:44.202 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.195+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 <== mon.2 v1:172.21.15.43:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 487425898 0 0) 0x7f0c78003520 con 0x7f0c8c1075d0 2024-03-21T13:28:44.202 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.195+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 --> v1:172.21.15.43:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0c8c1a77b0 con 0x7f0c8c1075d0 2024-03-21T13:28:44.202 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.195+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 <== mon.2 v1:172.21.15.43:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f0c88001930 con 0x7f0c8c1075d0 2024-03-21T13:28:44.203 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.195+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1399127755 0 0) 0x7f0c8c1a89f0 con 0x7f0c8c10df70 2024-03-21T13:28:44.203 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.195+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 --> v1:172.21.15.43:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0c78003520 con 0x7f0c8c10df70 2024-03-21T13:28:44.203 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.195+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f0c840013f0 con 0x7f0c8c10df70 2024-03-21T13:28:44.203 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.195+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 <== mon.2 v1:172.21.15.43:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 848327558 0 0) 0x7f0c8c1a77b0 con 0x7f0c8c1075d0 2024-03-21T13:28:44.203 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.195+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 >> v1:172.21.15.155:6789/0 conn(0x7f0c8c10a370 legacy=0x7f0c8c1a13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:28:44.203 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.195+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 >> v1:172.21.15.43:6789/0 conn(0x7f0c8c10df70 legacy=0x7f0c8c1a4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:28:44.204 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.195+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 --> v1:172.21.15.43:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c8c1a9c30 con 0x7f0c8c1075d0 2024-03-21T13:28:44.204 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.196+0000 7f0c93325640 1 -- 172.21.15.43:0/869710287 --> v1:172.21.15.43:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c8c1a67a0 con 0x7f0c8c1075d0 2024-03-21T13:28:44.204 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.196+0000 7f0c93325640 1 -- 172.21.15.43:0/869710287 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c8c1a6c70 con 0x7f0c8c1075d0 2024-03-21T13:28:44.204 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.196+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 <== mon.2 v1:172.21.15.43:6790/0 5 ==== config(25 keys) v1 ==== 1068+0+0 (unknown 3909832191 0 0) 0x7f0c88003aa0 con 0x7f0c8c1075d0 2024-03-21T13:28:44.204 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.196+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 <== mon.2 v1:172.21.15.43:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 1070066123 0 0) 0x7f0c88005c40 con 0x7f0c8c1075d0 2024-03-21T13:28:44.204 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.197+0000 7f0c93325640 1 -- 172.21.15.43:0/869710287 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c70005080 con 0x7f0c8c1075d0 2024-03-21T13:28:44.205 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.197+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 <== mon.2 v1:172.21.15.43:6790/0 7 ==== mgrmap(e 22) v1 ==== 92152+0+0 (unknown 2331634902 0 0) 0x7f0c88003610 con 0x7f0c8c1075d0 2024-03-21T13:28:44.205 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.198+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 <== mon.2 v1:172.21.15.43:6790/0 8 ==== osd_map(837..837 src has 251..837) v4 ==== 7798+0+0 (unknown 2580445982 0 0) 0x7f0c88089e90 con 0x7f0c8c1075d0 2024-03-21T13:28:44.205 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.198+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=838}) v3 -- 0x7f0c8c1a77b0 con 0x7f0c8c1075d0 2024-03-21T13:28:44.205 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.202+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 <== mon.2 v1:172.21.15.43:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (unknown 1092875540 0 3438036875) 0x7f0c88056ea0 con 0x7f0c8c1075d0 2024-03-21T13:28:44.389 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.388+0000 7f0c93325640 1 -- 172.21.15.43:0/869710287 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "osd pool delete", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pool2": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0c70005370 con 0x7f0c8c1075d0 2024-03-21T13:28:44.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:43 smithi155 ceph-mon[33846]: pgmap v1860: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:44.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:43 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3794097207' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pool2": "dfa6c75f-2043-4860-bf82-1f19a1404959", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T13:28:44.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:43 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:28:44.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:43 smithi155 ceph-mon[33846]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:28:44.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:43 smithi155 ceph-mon[33846]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:28:44.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:43 smithi043 ceph-mon[30062]: pgmap v1860: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:44.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:43 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3794097207' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pool2": "dfa6c75f-2043-4860-bf82-1f19a1404959", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T13:28:44.483 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:43 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:28:44.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:43 smithi043 ceph-mon[30062]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:28:44.484 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:43 smithi043 ceph-mon[30062]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:28:44.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:43 smithi043 ceph-mon[41699]: pgmap v1860: 188 pgs: 188 active+clean; 871 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-21T13:28:44.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:43 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3794097207' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pool2": "dfa6c75f-2043-4860-bf82-1f19a1404959", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T13:28:44.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:43 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T13:28:44.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:43 smithi043 ceph-mon[41699]: from='mgr.34493 v1:172.21.15.43:0/3518451155' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T13:28:44.484 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:43 smithi043 ceph-mon[41699]: from='mgr.34493 ' entity='mgr.y' 2024-03-21T13:28:44.994 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.993+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 <== mon.2 v1:172.21.15.43:6790/0 10 ==== osd_map(838..838 src has 251..838) v4 ==== 296+0+0 (unknown 3904009480 0 0) 0x7f0c88087c80 con 0x7f0c8c1075d0 2024-03-21T13:28:44.994 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:44.993+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 --> v1:172.21.15.43:6790/0 -- mon_subscribe({osdmap=839}) v3 -- 0x7f0c78003520 con 0x7f0c8c1075d0 2024-03-21T13:28:45.001 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:45.000+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 <== mon.2 v1:172.21.15.43:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pool2": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "yes_i_really_really_mean_it": true}]=0 pool 'c3eefea8-5703-4b79-a4c9-c6adb3b164cb' removed v838) v1 ==== 248+0+0 (unknown 1111882291 0 0) 0x7f0c8805a8f0 con 0x7f0c8c1075d0 2024-03-21T13:28:45.117 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:45.116+0000 7f0c93325640 1 -- 172.21.15.43:0/869710287 --> v1:172.21.15.43:6790/0 -- mon_command({"prefix": "osd pool delete", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pool2": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0c70005da0 con 0x7f0c8c1075d0 2024-03-21T13:28:45.118 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:45.117+0000 7f0c827fc640 1 -- 172.21.15.43:0/869710287 <== mon.2 v1:172.21.15.43:6790/0 12 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pool2": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "yes_i_really_really_mean_it": true}]=0 pool 'c3eefea8-5703-4b79-a4c9-c6adb3b164cb' does not exist v838) v1 ==== 255+0+0 (unknown 2729738279 0 0) 0x7f0c8805f1b0 con 0x7f0c8c1075d0 2024-03-21T13:28:45.118 INFO:tasks.workunit.client.0.smithi043.stderr:pool 'c3eefea8-5703-4b79-a4c9-c6adb3b164cb' does not exist 2024-03-21T13:28:45.120 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:45.119+0000 7f0c93325640 1 -- 172.21.15.43:0/869710287 >> v1:172.21.15.43:6800/4199387216 conn(0x7f0c7806f020 legacy=0x7f0c780714e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:28:45.120 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:45.119+0000 7f0c93325640 1 -- 172.21.15.43:0/869710287 >> v1:172.21.15.43:6790/0 conn(0x7f0c8c1075d0 legacy=0x7f0c8c106920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T13:28:45.120 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:45.119+0000 7f0c93325640 1 -- 172.21.15.43:0/869710287 shutdown_connections 2024-03-21T13:28:45.120 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:45.119+0000 7f0c93325640 1 -- 172.21.15.43:0/869710287 >> 172.21.15.43:0/869710287 conn(0x7f0c8c0fcf80 msgr2=0x7f0c8c0fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T13:28:45.121 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:45.119+0000 7f0c93325640 1 -- 172.21.15.43:0/869710287 shutdown_connections 2024-03-21T13:28:45.121 INFO:tasks.workunit.client.0.smithi043.stderr:2024-03-21T13:28:45.120+0000 7f0c93325640 1 -- 172.21.15.43:0/869710287 wait complete. 2024-03-21T13:28:45.129 INFO:tasks.workunit.client.0.smithi043.stderr:+ echo OK 2024-03-21T13:28:45.130 INFO:tasks.workunit.client.0.smithi043.stdout:OK 2024-03-21T13:28:45.131 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-21T13:28:45.131 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-21T13:28:45.200 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-03-21T13:28:45.200 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-21T13:28:45.264 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:44 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3794097207' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pool2": "dfa6c75f-2043-4860-bf82-1f19a1404959", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T13:28:45.264 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:44 smithi043 ceph-mon[30062]: osdmap e837: 8 total, 8 up, 8 in 2024-03-21T13:28:45.264 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:44 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/3794097207' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pool2": "dfa6c75f-2043-4860-bf82-1f19a1404959", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T13:28:45.264 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:44 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/869710287' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pool2": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T13:28:45.264 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:44 smithi043 ceph-mon[30062]: from='client.58997 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pool2": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T13:28:45.265 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:44 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3794097207' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pool2": "dfa6c75f-2043-4860-bf82-1f19a1404959", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T13:28:45.265 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:44 smithi043 ceph-mon[41699]: osdmap e837: 8 total, 8 up, 8 in 2024-03-21T13:28:45.265 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:44 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/3794097207' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pool2": "dfa6c75f-2043-4860-bf82-1f19a1404959", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T13:28:45.265 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:44 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/869710287' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pool2": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T13:28:45.265 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:44 smithi043 ceph-mon[41699]: from='client.58997 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pool2": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T13:28:45.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:44 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3794097207' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pool2": "dfa6c75f-2043-4860-bf82-1f19a1404959", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T13:28:45.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:44 smithi155 ceph-mon[33846]: osdmap e837: 8 total, 8 up, 8 in 2024-03-21T13:28:45.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:44 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/3794097207' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "dfa6c75f-2043-4860-bf82-1f19a1404959", "pool2": "dfa6c75f-2043-4860-bf82-1f19a1404959", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T13:28:45.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:44 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/869710287' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pool2": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T13:28:45.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:44 smithi155 ceph-mon[33846]: from='client.58997 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pool2": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T13:28:45.512 DEBUG:teuthology.parallel:result is None 2024-03-21T13:28:45.512 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-21T13:28:45.578 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-21T13:28:45.578 DEBUG:teuthology.orchestra.run.smithi043:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-03-21T13:28:45.637 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-03-21T13:28:45.637 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-21T13:28:45.647 INFO:tasks.cephadm:Teardown begin 2024-03-21T13:28:45.648 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T13:28:45.703 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T13:28:45.739 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-21T13:28:45.739 DEBUG:teuthology.orchestra.run.smithi043:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-21T13:28:45.757 DEBUG:teuthology.orchestra.run.smithi155:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-21T13:28:45.796 INFO:tasks.cephadm:Stopping all daemons... 2024-03-21T13:28:45.796 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-21T13:28:45.796 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mon.a 2024-03-21T13:28:46.133 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:45 smithi043 ceph-mon[41699]: pgmap v1862: 176 pgs: 176 active+clean; 698 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:28:46.133 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:45 smithi043 ceph-mon[41699]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-21T13:28:46.133 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:45 smithi043 ceph-mon[41699]: from='client.58997 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pool2": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T13:28:46.133 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:45 smithi043 ceph-mon[41699]: osdmap e838: 8 total, 8 up, 8 in 2024-03-21T13:28:46.133 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:45 smithi043 ceph-mon[41699]: from='client.? v1:172.21.15.43:0/869710287' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pool2": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T13:28:46.133 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:45 smithi043 ceph-mon[41699]: from='client.58997 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pool2": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T13:28:46.134 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:45 smithi043 systemd[1]: Stopping Ceph mon.a for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T13:28:46.134 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:45 smithi043 ceph-mon[30062]: pgmap v1862: 176 pgs: 176 active+clean; 698 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:28:46.134 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:45 smithi043 ceph-mon[30062]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-21T13:28:46.134 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:45 smithi043 ceph-mon[30062]: from='client.58997 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pool2": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T13:28:46.134 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:45 smithi043 ceph-mon[30062]: osdmap e838: 8 total, 8 up, 8 in 2024-03-21T13:28:46.134 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:45 smithi043 ceph-mon[30062]: from='client.? v1:172.21.15.43:0/869710287' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pool2": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T13:28:46.134 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:45 smithi043 ceph-mon[30062]: from='client.58997 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pool2": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T13:28:46.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:45 smithi155 ceph-mon[33846]: pgmap v1862: 176 pgs: 176 active+clean; 698 KiB data, 883 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-21T13:28:46.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:45 smithi155 ceph-mon[33846]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-21T13:28:46.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:45 smithi155 ceph-mon[33846]: from='client.58997 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pool2": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T13:28:46.457 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:45 smithi155 ceph-mon[33846]: osdmap e838: 8 total, 8 up, 8 in 2024-03-21T13:28:46.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:45 smithi155 ceph-mon[33846]: from='client.? v1:172.21.15.43:0/869710287' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pool2": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T13:28:46.458 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:45 smithi155 ceph-mon[33846]: from='client.58997 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "pool2": "c3eefea8-5703-4b79-a4c9-c6adb3b164cb", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T13:28:46.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:46 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-a[30039]: 2024-03-21T13:28:46.131+0000 7fd20deb9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T13:28:46.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:46 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-a[30039]: 2024-03-21T13:28:46.131+0000 7fd20deb9700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-21T13:28:46.481 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:46 smithi043 podman[160098]: 2024-03-21 13:28:46.173973969 +0000 UTC m=+0.192456257 container died c6b9cf84086b2ed440fd90cee2d37a154e05c6c0f7b3fc2f3eaeff8099c00d7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-21T13:28:47.098 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:46 smithi043 podman[160098]: 2024-03-21 13:28:46.822596133 +0000 UTC m=+0.841078472 container cleanup c6b9cf84086b2ed440fd90cee2d37a154e05c6c0f7b3fc2f3eaeff8099c00d7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-a, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-21T13:28:47.098 INFO:journalctl@ceph.mon.a.smithi043.stdout:Mar 21 13:28:46 smithi043 bash[160098]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-a 2024-03-21T13:28:47.254 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mon.a.service' 2024-03-21T13:28:47.293 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T13:28:47.293 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-21T13:28:47.293 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-03-21T13:28:47.294 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mon.c 2024-03-21T13:28:47.638 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:47 smithi043 systemd[1]: Stopping Ceph mon.c for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T13:28:47.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:47 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-c[41675]: 2024-03-21T13:28:47.637+0000 7fa896ab3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T13:28:47.983 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:47 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-c[41675]: 2024-03-21T13:28:47.637+0000 7fa896ab3700 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-03-21T13:28:47.984 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:47 smithi043 podman[160232]: 2024-03-21 13:28:47.79155298 +0000 UTC m=+0.279266578 container died 52be75e7f9260000c89810f899872047387d423fe663942d4acbbb32974ce509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-c, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-21T13:28:47.984 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:28:47 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:13:28:47] ENGINE Bus STOPPING 2024-03-21T13:28:47.984 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:28:47 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:13:28:47] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-21T13:28:47.984 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:28:47 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:13:28:47] ENGINE Bus STOPPED 2024-03-21T13:28:47.984 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:28:47 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:13:28:47] ENGINE Bus STARTING 2024-03-21T13:28:47.984 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:28:47 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:13:28:47] ENGINE Serving on http://:::9283 2024-03-21T13:28:47.984 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:28:47 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: [21/Mar/2024:13:28:47] ENGINE Bus STARTED 2024-03-21T13:28:48.285 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:28:48 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: ::ffff:172.21.15.155 - - [21/Mar/2024:13:28:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-21T13:28:48.643 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:48 smithi043 podman[160232]: 2024-03-21 13:28:48.28638826 +0000 UTC m=+0.774101860 container cleanup 52be75e7f9260000c89810f899872047387d423fe663942d4acbbb32974ce509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-03-21T13:28:48.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:48 smithi043 bash[160232]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-c 2024-03-21T13:28:48.644 INFO:journalctl@ceph.mon.c.smithi043.stdout:Mar 21 13:28:48 smithi043 podman[160260]: 2024-03-21 13:28:48.520879887 +0000 UTC m=+0.728538209 container remove 52be75e7f9260000c89810f899872047387d423fe663942d4acbbb32974ce509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-c, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, io.buildah.version=1.33.5) 2024-03-21T13:28:48.654 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mon.c.service' 2024-03-21T13:28:48.690 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T13:28:48.690 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-03-21T13:28:48.690 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-03-21T13:28:48.690 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mon.b 2024-03-21T13:28:49.071 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:48 smithi155 systemd[1]: Stopping Ceph mon.b for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T13:28:49.422 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-b[33823]: 2024-03-21T13:28:49.070+0000 7fcff8bcd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T13:28:49.422 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-b[33823]: 2024-03-21T13:28:49.070+0000 7fcff8bcd700 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-03-21T13:28:49.422 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:49 smithi155 podman[79756]: 2024-03-21 13:28:49.102282044 +0000 UTC m=+0.208603605 container died 1a67652954ff164ad8be73e46fb90dcd0c12e9f2ca2161eb47c2266c21fdaf4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-b, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-21T13:28:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:49 smithi155 podman[79756]: 2024-03-21 13:28:49.619243617 +0000 UTC m=+0.725565178 container cleanup 1a67652954ff164ad8be73e46fb90dcd0c12e9f2ca2161eb47c2266c21fdaf4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319) 2024-03-21T13:28:49.707 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:49 smithi155 bash[79756]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-b 2024-03-21T13:28:50.079 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mon.b.service' 2024-03-21T13:28:50.104 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:28:50 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug there is no tcmu-runner data available 2024-03-21T13:28:50.104 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:49 smithi155 podman[79775]: 2024-03-21 13:28:49.85418402 +0000 UTC m=+0.756845161 container remove 1a67652954ff164ad8be73e46fb90dcd0c12e9f2ca2161eb47c2266c21fdaf4c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mon-b, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-21T13:28:50.104 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:50 smithi155 systemd[1]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mon.b.service: Deactivated successfully. 2024-03-21T13:28:50.104 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:50 smithi155 systemd[1]: Stopped Ceph mon.b for c06c8d1a-e780-11ee-95cd-87774f69a715. 2024-03-21T13:28:50.104 INFO:journalctl@ceph.mon.b.smithi155.stdout:Mar 21 13:28:50 smithi155 systemd[1]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mon.b.service: Consumed 33.263s CPU time. 2024-03-21T13:28:50.138 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T13:28:50.138 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-03-21T13:28:50.138 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-21T13:28:50.138 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mgr.y 2024-03-21T13:28:50.442 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:28:50 smithi043 systemd[1]: Stopping Ceph mgr.y for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T13:28:50.734 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:28:50 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T13:28:50.441+0000 7f8bb7c89700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T13:28:50.734 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:28:50 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y[30323]: 2024-03-21T13:28:50.441+0000 7f8bb7c89700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-21T13:28:50.734 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:28:50 smithi043 podman[160376]: 2024-03-21 13:28:50.474199754 +0000 UTC m=+0.140620551 container died a65d78811d7799862d0e8e9234e6ad3abe4cd40b6af9152fb7d496f3b4461778 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-21T13:28:51.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:28:50 smithi043 podman[160376]: 2024-03-21 13:28:50.974504691 +0000 UTC m=+0.640925543 container cleanup a65d78811d7799862d0e8e9234e6ad3abe4cd40b6af9152fb7d496f3b4461778 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:960fbbe4645c2becc8084669be194c95183c7753, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-21T13:28:51.233 INFO:journalctl@ceph.mgr.y.smithi043.stdout:Mar 21 13:28:50 smithi043 bash[160376]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-y 2024-03-21T13:28:51.404 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mgr.y.service' 2024-03-21T13:28:51.441 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T13:28:51.441 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-21T13:28:51.441 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-21T13:28:51.441 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mgr.x 2024-03-21T13:28:51.791 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 13:28:51 smithi155 systemd[1]: Stopping Ceph mgr.x for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T13:28:52.082 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 13:28:51 smithi155 podman[79890]: 2024-03-21 13:28:51.790510925 +0000 UTC m=+0.153007696 container died df012bc317847b9b083d81f4a135fb3c3eb88130ccd4ed684696e3d6143d6860 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-21T13:28:52.424 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 13:28:52 smithi155 podman[79890]: 2024-03-21 13:28:52.182000479 +0000 UTC m=+0.544497242 container cleanup df012bc317847b9b083d81f4a135fb3c3eb88130ccd4ed684696e3d6143d6860 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-21T13:28:52.425 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Mar 21 13:28:52 smithi155 bash[79890]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-mgr-x 2024-03-21T13:28:52.553 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@mgr.x.service' 2024-03-21T13:28:52.591 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T13:28:52.591 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-21T13:28:52.591 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-21T13:28:52.591 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@osd.0 2024-03-21T13:28:52.897 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 21 13:28:52 smithi043 systemd[1]: Stopping Ceph osd.0 for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T13:28:53.233 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 21 13:28:52 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-0[46126]: 2024-03-21T13:28:52.896+0000 7f615f803700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T13:28:53.234 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 21 13:28:52 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-0[46126]: 2024-03-21T13:28:52.896+0000 7f615f803700 -1 osd.0 838 *** Got signal Terminated *** 2024-03-21T13:28:53.234 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 21 13:28:52 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-0[46126]: 2024-03-21T13:28:52.896+0000 7f615f803700 -1 osd.0 838 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T13:28:58.597 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 21 13:28:58 smithi043 podman[160509]: 2024-03-21 13:28:58.304361096 +0000 UTC m=+5.518488416 container died 9387aa6dc2f949d911ac36f4826b32c93cf7deb205c9c9fb2802b6a176ee34f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True) 2024-03-21T13:28:58.979 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 21 13:28:58 smithi043 podman[160509]: 2024-03-21 13:28:58.695063996 +0000 UTC m=+5.909191306 container cleanup 9387aa6dc2f949d911ac36f4826b32c93cf7deb205c9c9fb2802b6a176ee34f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-21T13:28:58.979 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 21 13:28:58 smithi043 bash[160509]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-0 2024-03-21T13:28:59.233 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 21 13:28:58 smithi043 podman[160531]: 2024-03-21 13:28:58.979880877 +0000 UTC m=+0.669156529 container remove 9387aa6dc2f949d911ac36f4826b32c93cf7deb205c9c9fb2802b6a176ee34f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-21T13:28:59.632 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 21 13:28:59 smithi043 podman[160624]: 2024-03-21 13:28:59.220687592 +0000 UTC m=+0.019469669 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93 2024-03-21T13:28:59.632 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 21 13:28:59 smithi043 podman[160624]: 2024-03-21 13:28:59.339144292 +0000 UTC m=+0.137926414 container create 1153da6ec31f9a5b6526d1e8ff32e15c483a7568c73f45fde1450f1824f837fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-0-deactivate, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-21T13:28:59.633 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 21 13:28:59 smithi043 podman[160624]: 2024-03-21 13:28:59.559381699 +0000 UTC m=+0.358163776 container init 1153da6ec31f9a5b6526d1e8ff32e15c483a7568c73f45fde1450f1824f837fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T13:28:59.633 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 21 13:28:59 smithi043 podman[160624]: 2024-03-21 13:28:59.565235982 +0000 UTC m=+0.364018059 container start 1153da6ec31f9a5b6526d1e8ff32e15c483a7568c73f45fde1450f1824f837fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T13:28:59.633 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 21 13:28:59 smithi043 podman[160624]: 2024-03-21 13:28:59.633108367 +0000 UTC m=+0.431890471 container attach 1153da6ec31f9a5b6526d1e8ff32e15c483a7568c73f45fde1450f1824f837fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, ceph=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T13:28:59.983 INFO:journalctl@ceph.osd.0.smithi043.stdout:Mar 21 13:28:59 smithi043 podman[160624]: 2024-03-21 13:28:59.865109439 +0000 UTC m=+0.663891518 container died 1153da6ec31f9a5b6526d1e8ff32e15c483a7568c73f45fde1450f1824f837fa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-21T13:29:00.531 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@osd.0.service' 2024-03-21T13:29:00.572 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T13:29:00.572 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-21T13:29:00.572 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-21T13:29:00.572 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@osd.1 2024-03-21T13:29:00.925 INFO:journalctl@ceph.osd.1.smithi043.stdout:Mar 21 13:29:00 smithi043 systemd[1]: Stopping Ceph osd.1 for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T13:29:01.233 INFO:journalctl@ceph.osd.1.smithi043.stdout:Mar 21 13:29:00 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-1[52242]: 2024-03-21T13:29:00.924+0000 7fd9f17fb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T13:29:01.234 INFO:journalctl@ceph.osd.1.smithi043.stdout:Mar 21 13:29:00 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-1[52242]: 2024-03-21T13:29:00.924+0000 7fd9f17fb700 -1 osd.1 838 *** Got signal Terminated *** 2024-03-21T13:29:01.234 INFO:journalctl@ceph.osd.1.smithi043.stdout:Mar 21 13:29:00 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-1[52242]: 2024-03-21T13:29:00.924+0000 7fd9f17fb700 -1 osd.1 838 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T13:29:06.566 INFO:journalctl@ceph.osd.1.smithi043.stdout:Mar 21 13:29:06 smithi043 podman[160758]: 2024-03-21 13:29:06.253873335 +0000 UTC m=+5.452695839 container died 22be0baec1c25bf0a8dfd5fdbdb369d9380b15dde364428d8d79e189179c3bf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-1, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True) 2024-03-21T13:29:06.849 INFO:journalctl@ceph.osd.1.smithi043.stdout:Mar 21 13:29:06 smithi043 podman[160758]: 2024-03-21 13:29:06.656771766 +0000 UTC m=+5.855594268 container cleanup 22be0baec1c25bf0a8dfd5fdbdb369d9380b15dde364428d8d79e189179c3bf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-1, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2) 2024-03-21T13:29:06.849 INFO:journalctl@ceph.osd.1.smithi043.stdout:Mar 21 13:29:06 smithi043 bash[160758]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-1 2024-03-21T13:29:07.233 INFO:journalctl@ceph.osd.1.smithi043.stdout:Mar 21 13:29:06 smithi043 podman[160780]: 2024-03-21 13:29:06.849711588 +0000 UTC m=+0.588060409 container remove 22be0baec1c25bf0a8dfd5fdbdb369d9380b15dde364428d8d79e189179c3bf4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-1, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T13:29:07.578 INFO:journalctl@ceph.osd.1.smithi043.stdout:Mar 21 13:29:07 smithi043 podman[160879]: 2024-03-21 13:29:07.156906712 +0000 UTC m=+0.018263699 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93 2024-03-21T13:29:07.578 INFO:journalctl@ceph.osd.1.smithi043.stdout:Mar 21 13:29:07 smithi043 podman[160879]: 2024-03-21 13:29:07.258818759 +0000 UTC m=+0.120175779 container create ee3996583a144bf71606dc213d9f8c721ca7739becd5a725687c9735770571e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T13:29:07.578 INFO:journalctl@ceph.osd.1.smithi043.stdout:Mar 21 13:29:07 smithi043 podman[160879]: 2024-03-21 13:29:07.48655031 +0000 UTC m=+0.347907289 container init ee3996583a144bf71606dc213d9f8c721ca7739becd5a725687c9735770571e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-21T13:29:07.578 INFO:journalctl@ceph.osd.1.smithi043.stdout:Mar 21 13:29:07 smithi043 podman[160879]: 2024-03-21 13:29:07.492060652 +0000 UTC m=+0.353417631 container start ee3996583a144bf71606dc213d9f8c721ca7739becd5a725687c9735770571e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319) 2024-03-21T13:29:07.578 INFO:journalctl@ceph.osd.1.smithi043.stdout:Mar 21 13:29:07 smithi043 podman[160879]: 2024-03-21 13:29:07.577741276 +0000 UTC m=+0.439098255 container attach ee3996583a144bf71606dc213d9f8c721ca7739becd5a725687c9735770571e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T13:29:07.986 INFO:journalctl@ceph.osd.1.smithi043.stdout:Mar 21 13:29:07 smithi043 podman[160879]: 2024-03-21 13:29:07.788445188 +0000 UTC m=+0.649802166 container died ee3996583a144bf71606dc213d9f8c721ca7739becd5a725687c9735770571e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T13:29:08.384 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@osd.1.service' 2024-03-21T13:29:08.425 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T13:29:08.425 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-21T13:29:08.425 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-21T13:29:08.425 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@osd.2 2024-03-21T13:29:08.733 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 13:29:08 smithi043 systemd[1]: Stopping Ceph osd.2 for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T13:29:09.233 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 13:29:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-2[58226]: 2024-03-21T13:29:08.757+0000 7f6a401e1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T13:29:09.234 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 13:29:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-2[58226]: 2024-03-21T13:29:08.757+0000 7f6a401e1700 -1 osd.2 838 *** Got signal Terminated *** 2024-03-21T13:29:09.234 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 13:29:08 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-2[58226]: 2024-03-21T13:29:08.757+0000 7f6a401e1700 -1 osd.2 838 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T13:29:14.535 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 13:29:14 smithi043 podman[161010]: 2024-03-21 13:29:14.258846709 +0000 UTC m=+5.613230975 container died 52f0c037401e548dd959dcb991ef04ef6434cc9311e6b874a67dbbd755a90a92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-2, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T13:29:14.893 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 13:29:14 smithi043 podman[161010]: 2024-03-21 13:29:14.634016255 +0000 UTC m=+5.988400544 container cleanup 52f0c037401e548dd959dcb991ef04ef6434cc9311e6b874a67dbbd755a90a92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True) 2024-03-21T13:29:14.893 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 13:29:14 smithi043 bash[161010]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-2 2024-03-21T13:29:15.205 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 13:29:14 smithi043 podman[161044]: 2024-03-21 13:29:14.89339063 +0000 UTC m=+0.631605005 container remove 52f0c037401e548dd959dcb991ef04ef6434cc9311e6b874a67dbbd755a90a92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-21T13:29:15.484 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 13:29:15 smithi043 podman[161141]: 2024-03-21 13:29:15.108745681 +0000 UTC m=+0.017803225 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93 2024-03-21T13:29:15.484 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 13:29:15 smithi043 podman[161141]: 2024-03-21 13:29:15.227674236 +0000 UTC m=+0.136731780 container create b7263e1f3c21e0429714bc4e35b9324e1ffab4bd8ed97423d17643401a014881 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe) 2024-03-21T13:29:15.484 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 13:29:15 smithi043 podman[161141]: 2024-03-21 13:29:15.422006156 +0000 UTC m=+0.331063714 container init b7263e1f3c21e0429714bc4e35b9324e1ffab4bd8ed97423d17643401a014881 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-03-21T13:29:15.484 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 13:29:15 smithi043 podman[161141]: 2024-03-21 13:29:15.427540486 +0000 UTC m=+0.336598038 container start b7263e1f3c21e0429714bc4e35b9324e1ffab4bd8ed97423d17643401a014881 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-03-21T13:29:15.983 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 13:29:15 smithi043 podman[161141]: 2024-03-21 13:29:15.513297491 +0000 UTC m=+0.422355037 container attach b7263e1f3c21e0429714bc4e35b9324e1ffab4bd8ed97423d17643401a014881 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.schema-version=1.0) 2024-03-21T13:29:15.983 INFO:journalctl@ceph.osd.2.smithi043.stdout:Mar 21 13:29:15 smithi043 podman[161141]: 2024-03-21 13:29:15.73238024 +0000 UTC m=+0.641437775 container died b7263e1f3c21e0429714bc4e35b9324e1ffab4bd8ed97423d17643401a014881 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-21T13:29:16.349 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@osd.2.service' 2024-03-21T13:29:16.391 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T13:29:16.391 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-21T13:29:16.392 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-21T13:29:16.392 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@osd.3 2024-03-21T13:29:16.714 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 21 13:29:16 smithi043 systemd[1]: Stopping Ceph osd.3 for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T13:29:16.983 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 21 13:29:16 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-3[64567]: 2024-03-21T13:29:16.714+0000 7f76818ba700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T13:29:16.983 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 21 13:29:16 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-3[64567]: 2024-03-21T13:29:16.714+0000 7f76818ba700 -1 osd.3 838 *** Got signal Terminated *** 2024-03-21T13:29:16.984 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 21 13:29:16 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-3[64567]: 2024-03-21T13:29:16.714+0000 7f76818ba700 -1 osd.3 838 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T13:29:19.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:19 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:19.349+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:19.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:19 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:19.100+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:20.379 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:20 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:20.102+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:20.707 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:20 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:20.378+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:21.426 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 13:29:21 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-4[39286]: 2024-03-21T13:29:21.117+0000 7f366b944700 -1 osd.4 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.807717+0000 front 2024-03-21T13:28:57.807824+0000 (oldest deadline 2024-03-21T13:29:20.107556+0000) 2024-03-21T13:29:21.426 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:20 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:20.967+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:56.549589+0000 front 2024-03-21T13:28:56.549461+0000 (oldest deadline 2024-03-21T13:29:20.649493+0000) 2024-03-21T13:29:21.426 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:21 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:21.068+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:21.707 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:21 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:21.426+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:22.353 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 21 13:29:22 smithi043 podman[161274]: 2024-03-21 13:29:22.101332342 +0000 UTC m=+5.485281996 container died 30528db50ccec63ed9e80d438b4d5ff7491570747aaf38bdd35bae06ebf03547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-3, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, ceph=True) 2024-03-21T13:29:22.433 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 13:29:22 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-4[39286]: 2024-03-21T13:29:22.152+0000 7f366b944700 -1 osd.4 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.807717+0000 front 2024-03-21T13:28:57.807824+0000 (oldest deadline 2024-03-21T13:29:20.107556+0000) 2024-03-21T13:29:22.434 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:21 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:21.959+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:56.549589+0000 front 2024-03-21T13:28:56.549461+0000 (oldest deadline 2024-03-21T13:29:20.649493+0000) 2024-03-21T13:29:22.434 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:22 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:22.059+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:22.707 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:22 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:22.434+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:22.733 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 21 13:29:22 smithi043 podman[161274]: 2024-03-21 13:29:22.410914307 +0000 UTC m=+5.794864005 container cleanup 30528db50ccec63ed9e80d438b4d5ff7491570747aaf38bdd35bae06ebf03547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, maintainer=Guillaume Abrioux ) 2024-03-21T13:29:22.733 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 21 13:29:22 smithi043 bash[161274]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-3 2024-03-21T13:29:22.733 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 21 13:29:22 smithi043 podman[161296]: 2024-03-21 13:29:22.570790591 +0000 UTC m=+0.470084219 container remove 30528db50ccec63ed9e80d438b4d5ff7491570747aaf38bdd35bae06ebf03547 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-3, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-21T13:29:23.233 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 21 13:29:22 smithi043 podman[161395]: 2024-03-21 13:29:22.875599378 +0000 UTC m=+0.018364811 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93 2024-03-21T13:29:23.234 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 21 13:29:22 smithi043 podman[161395]: 2024-03-21 13:29:22.979944259 +0000 UTC m=+0.122709731 container create 147cd98c6e6d7cac68d441cdef5276a26240549fcea2bd46b0c1e77b64d0584f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-21T13:29:23.234 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 21 13:29:23 smithi043 podman[161395]: 2024-03-21 13:29:23.190648549 +0000 UTC m=+0.333413988 container init 147cd98c6e6d7cac68d441cdef5276a26240549fcea2bd46b0c1e77b64d0584f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T13:29:23.234 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 21 13:29:23 smithi043 podman[161395]: 2024-03-21 13:29:23.196029369 +0000 UTC m=+0.338794797 container start 147cd98c6e6d7cac68d441cdef5276a26240549fcea2bd46b0c1e77b64d0584f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T13:29:23.392 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 13:29:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-4[39286]: 2024-03-21T13:29:23.176+0000 7f366b944700 -1 osd.4 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.807717+0000 front 2024-03-21T13:28:57.807824+0000 (oldest deadline 2024-03-21T13:29:20.107556+0000) 2024-03-21T13:29:23.392 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:22 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:22.973+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:56.549589+0000 front 2024-03-21T13:28:56.549461+0000 (oldest deadline 2024-03-21T13:29:20.649493+0000) 2024-03-21T13:29:23.393 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:23.075+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:23.707 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:23.392+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:23.733 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 21 13:29:23 smithi043 podman[161395]: 2024-03-21 13:29:23.28197597 +0000 UTC m=+0.424741408 container attach 147cd98c6e6d7cac68d441cdef5276a26240549fcea2bd46b0c1e77b64d0584f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T13:29:23.734 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 21 13:29:23 smithi043 podman[161395]: 2024-03-21 13:29:23.490181845 +0000 UTC m=+0.632947286 container died 147cd98c6e6d7cac68d441cdef5276a26240549fcea2bd46b0c1e77b64d0584f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-03-21T13:29:24.062 INFO:journalctl@ceph.osd.3.smithi043.stdout:Mar 21 13:29:24 smithi043 podman[161441]: 2024-03-21 13:29:24.060230701 +0000 UTC m=+0.565049798 container remove 147cd98c6e6d7cac68d441cdef5276a26240549fcea2bd46b0c1e77b64d0584f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T13:29:24.076 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@osd.3.service' 2024-03-21T13:29:24.119 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T13:29:24.119 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-21T13:29:24.119 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-21T13:29:24.119 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@osd.4 2024-03-21T13:29:24.412 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 13:29:24 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-4[39286]: 2024-03-21T13:29:24.131+0000 7f366b944700 -1 osd.4 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.807717+0000 front 2024-03-21T13:28:57.807824+0000 (oldest deadline 2024-03-21T13:29:20.107556+0000) 2024-03-21T13:29:24.412 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 13:29:24 smithi155 systemd[1]: Stopping Ceph osd.4 for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T13:29:24.412 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:23 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:23.967+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:56.549589+0000 front 2024-03-21T13:28:56.549461+0000 (oldest deadline 2024-03-21T13:29:20.649493+0000) 2024-03-21T13:29:24.412 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:24 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:24.067+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:24.707 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:24 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:24.411+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:24.707 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 13:29:24 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-4[39286]: 2024-03-21T13:29:24.461+0000 7f366f350700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T13:29:24.707 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 13:29:24 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-4[39286]: 2024-03-21T13:29:24.461+0000 7f366f350700 -1 osd.4 838 *** Got signal Terminated *** 2024-03-21T13:29:24.707 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 13:29:24 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-4[39286]: 2024-03-21T13:29:24.462+0000 7f366f350700 -1 osd.4 838 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T13:29:25.376 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 13:29:25 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-4[39286]: 2024-03-21T13:29:25.150+0000 7f366b944700 -1 osd.4 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.807717+0000 front 2024-03-21T13:28:57.807824+0000 (oldest deadline 2024-03-21T13:29:20.107556+0000) 2024-03-21T13:29:25.376 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:24 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:24.974+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:56.549589+0000 front 2024-03-21T13:28:56.549461+0000 (oldest deadline 2024-03-21T13:29:20.649493+0000) 2024-03-21T13:29:25.376 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:25 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:25.032+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:25.707 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:25 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:25.375+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:26.333 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 13:29:26 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-4[39286]: 2024-03-21T13:29:26.134+0000 7f366b944700 -1 osd.4 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.807717+0000 front 2024-03-21T13:28:57.807824+0000 (oldest deadline 2024-03-21T13:29:20.107556+0000) 2024-03-21T13:29:26.334 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:25 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:25.956+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:56.549589+0000 front 2024-03-21T13:28:56.549461+0000 (oldest deadline 2024-03-21T13:29:20.649493+0000) 2024-03-21T13:29:26.334 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:26 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:26.039+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:26.707 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:26 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:26.332+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:27.207 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:27 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:27.027+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:27.207 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 13:29:27 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-4[39286]: 2024-03-21T13:29:27.102+0000 7f366b944700 -1 osd.4 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.807717+0000 front 2024-03-21T13:28:57.807824+0000 (oldest deadline 2024-03-21T13:29:20.107556+0000) 2024-03-21T13:29:27.207 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:26 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:26.912+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:56.549589+0000 front 2024-03-21T13:28:56.549461+0000 (oldest deadline 2024-03-21T13:29:20.649493+0000) 2024-03-21T13:29:27.707 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:27 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:27.301+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:28.207 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:28 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:28.058+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:28.207 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 13:29:28 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-4[39286]: 2024-03-21T13:29:28.117+0000 7f366b944700 -1 osd.4 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.807717+0000 front 2024-03-21T13:28:57.807824+0000 (oldest deadline 2024-03-21T13:29:20.107556+0000) 2024-03-21T13:29:28.207 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:27 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:27.917+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:56.549589+0000 front 2024-03-21T13:28:56.549461+0000 (oldest deadline 2024-03-21T13:29:20.649493+0000) 2024-03-21T13:29:28.208 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:27 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:27.917+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:04.750466+0000 front 2024-03-21T13:29:04.750495+0000 (oldest deadline 2024-03-21T13:29:27.650390+0000) 2024-03-21T13:29:28.707 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:28 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:28.279+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:28.707 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:28 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:28.279+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.681769+0000 front 2024-03-21T13:29:02.681923+0000 (oldest deadline 2024-03-21T13:29:27.981820+0000) 2024-03-21T13:29:29.207 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:29.075+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:29.207 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:29.075+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.905348+0000 front 2024-03-21T13:29:02.905252+0000 (oldest deadline 2024-03-21T13:29:28.205380+0000) 2024-03-21T13:29:29.207 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 13:29:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-4[39286]: 2024-03-21T13:29:29.155+0000 7f366b944700 -1 osd.4 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.807717+0000 front 2024-03-21T13:28:57.807824+0000 (oldest deadline 2024-03-21T13:29:20.107556+0000) 2024-03-21T13:29:29.208 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 13:29:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-4[39286]: 2024-03-21T13:29:29.155+0000 7f366b944700 -1 osd.4 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:05.909056+0000 front 2024-03-21T13:29:05.909088+0000 (oldest deadline 2024-03-21T13:29:28.208912+0000) 2024-03-21T13:29:29.208 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:28 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:28.952+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:56.549589+0000 front 2024-03-21T13:28:56.549461+0000 (oldest deadline 2024-03-21T13:29:20.649493+0000) 2024-03-21T13:29:29.208 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:28 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:28.952+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:04.750466+0000 front 2024-03-21T13:29:04.750495+0000 (oldest deadline 2024-03-21T13:29:27.650390+0000) 2024-03-21T13:29:29.707 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:29.249+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:29.707 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:29.249+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.681769+0000 front 2024-03-21T13:29:02.681923+0000 (oldest deadline 2024-03-21T13:29:27.981820+0000) 2024-03-21T13:29:30.093 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 13:29:29 smithi155 podman[80040]: 2024-03-21 13:29:29.801765001 +0000 UTC m=+5.473526845 container died 9e17d02c50b9d3ccd97bbc00a4f5491f69d1f3ac5a168f0456f28652779df1d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True) 2024-03-21T13:29:30.093 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:29.945+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:56.549589+0000 front 2024-03-21T13:28:56.549461+0000 (oldest deadline 2024-03-21T13:29:20.649493+0000) 2024-03-21T13:29:30.093 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:29 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:29.945+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:04.750466+0000 front 2024-03-21T13:29:04.750495+0000 (oldest deadline 2024-03-21T13:29:27.650390+0000) 2024-03-21T13:29:30.094 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:30 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:30.027+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:30.094 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:30 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:30.027+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.905348+0000 front 2024-03-21T13:29:02.905252+0000 (oldest deadline 2024-03-21T13:29:28.205380+0000) 2024-03-21T13:29:30.443 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 13:29:30 smithi155 podman[80040]: 2024-03-21 13:29:30.192569375 +0000 UTC m=+5.864331211 container cleanup 9e17d02c50b9d3ccd97bbc00a4f5491f69d1f3ac5a168f0456f28652779df1d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-4, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-21T13:29:30.444 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 13:29:30 smithi155 bash[80040]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-4 2024-03-21T13:29:30.444 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:30 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:30.231+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:30.444 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:30 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:30.231+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.681769+0000 front 2024-03-21T13:29:02.681923+0000 (oldest deadline 2024-03-21T13:29:27.981820+0000) 2024-03-21T13:29:30.707 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 13:29:30 smithi155 podman[80062]: 2024-03-21 13:29:30.443916195 +0000 UTC m=+0.642160405 container remove 9e17d02c50b9d3ccd97bbc00a4f5491f69d1f3ac5a168f0456f28652779df1d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-4, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-21T13:29:31.022 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:30 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:30.927+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:56.549589+0000 front 2024-03-21T13:28:56.549461+0000 (oldest deadline 2024-03-21T13:29:20.649493+0000) 2024-03-21T13:29:31.022 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:30 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:30.927+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:04.750466+0000 front 2024-03-21T13:29:04.750495+0000 (oldest deadline 2024-03-21T13:29:27.650390+0000) 2024-03-21T13:29:31.023 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 13:29:30 smithi155 podman[80156]: 2024-03-21 13:29:30.665581586 +0000 UTC m=+0.017276339 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93 2024-03-21T13:29:31.023 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 13:29:30 smithi155 podman[80156]: 2024-03-21 13:29:30.76980968 +0000 UTC m=+0.121504422 container create 1c8832761d9a21025393a98b8144912cd26734c11a0d2bcd59e7c2b9e219d051 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=) 2024-03-21T13:29:31.023 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 13:29:30 smithi155 podman[80156]: 2024-03-21 13:29:30.989625058 +0000 UTC m=+0.341319814 container init 1c8832761d9a21025393a98b8144912cd26734c11a0d2bcd59e7c2b9e219d051 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS) 2024-03-21T13:29:31.023 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 13:29:30 smithi155 podman[80156]: 2024-03-21 13:29:30.995427815 +0000 UTC m=+0.347122672 container start 1c8832761d9a21025393a98b8144912cd26734c11a0d2bcd59e7c2b9e219d051 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-21T13:29:31.292 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 13:29:31 smithi155 podman[80156]: 2024-03-21 13:29:31.08915961 +0000 UTC m=+0.440854372 container attach 1c8832761d9a21025393a98b8144912cd26734c11a0d2bcd59e7c2b9e219d051 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-21T13:29:31.292 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:31 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:31.021+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:31.292 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:31 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:31.021+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.905348+0000 front 2024-03-21T13:29:02.905252+0000 (oldest deadline 2024-03-21T13:29:28.205380+0000) 2024-03-21T13:29:31.293 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:31 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:31.220+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:31.293 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:31 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:31.220+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.681769+0000 front 2024-03-21T13:29:02.681923+0000 (oldest deadline 2024-03-21T13:29:27.981820+0000) 2024-03-21T13:29:31.600 INFO:journalctl@ceph.osd.4.smithi155.stdout:Mar 21 13:29:31 smithi155 podman[80156]: 2024-03-21 13:29:31.292648669 +0000 UTC m=+0.644343420 container died 1c8832761d9a21025393a98b8144912cd26734c11a0d2bcd59e7c2b9e219d051 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-21T13:29:31.914 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@osd.4.service' 2024-03-21T13:29:31.954 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T13:29:31.954 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-21T13:29:31.955 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-21T13:29:31.955 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@osd.5 2024-03-21T13:29:32.193 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:31 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:31.936+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:56.549589+0000 front 2024-03-21T13:28:56.549461+0000 (oldest deadline 2024-03-21T13:29:20.649493+0000) 2024-03-21T13:29:32.193 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:31 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:31.936+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:04.750466+0000 front 2024-03-21T13:29:04.750495+0000 (oldest deadline 2024-03-21T13:29:27.650390+0000) 2024-03-21T13:29:32.193 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:32 smithi155 systemd[1]: Stopping Ceph osd.5 for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T13:29:32.193 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:32 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:32.031+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:32.193 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:32 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:32.031+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.905348+0000 front 2024-03-21T13:29:02.905252+0000 (oldest deadline 2024-03-21T13:29:28.205380+0000) 2024-03-21T13:29:32.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:32 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:32.191+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:32.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:32 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:32.191+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.681769+0000 front 2024-03-21T13:29:02.681923+0000 (oldest deadline 2024-03-21T13:29:27.981820+0000) 2024-03-21T13:29:32.457 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:32 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:32.305+0000 7f43ad345700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T13:29:32.457 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:32 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:32.305+0000 7f43ad345700 -1 osd.5 838 *** Got signal Terminated *** 2024-03-21T13:29:32.457 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:32 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:32.305+0000 7f43ad345700 -1 osd.5 838 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T13:29:33.200 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:32 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:32.948+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:56.549589+0000 front 2024-03-21T13:28:56.549461+0000 (oldest deadline 2024-03-21T13:29:20.649493+0000) 2024-03-21T13:29:33.200 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:32 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:32.948+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:04.750466+0000 front 2024-03-21T13:29:04.750495+0000 (oldest deadline 2024-03-21T13:29:27.650390+0000) 2024-03-21T13:29:33.200 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:33 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:33.058+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:33.200 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:33 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:33.058+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.905348+0000 front 2024-03-21T13:29:02.905252+0000 (oldest deadline 2024-03-21T13:29:28.205380+0000) 2024-03-21T13:29:33.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:33 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:33.199+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:33.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:33 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:33.199+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.681769+0000 front 2024-03-21T13:29:02.681923+0000 (oldest deadline 2024-03-21T13:29:27.981820+0000) 2024-03-21T13:29:34.224 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:33 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:33.970+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:56.549589+0000 front 2024-03-21T13:28:56.549461+0000 (oldest deadline 2024-03-21T13:29:20.649493+0000) 2024-03-21T13:29:34.224 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:33 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:33.970+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:04.750466+0000 front 2024-03-21T13:29:04.750495+0000 (oldest deadline 2024-03-21T13:29:27.650390+0000) 2024-03-21T13:29:34.224 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:34 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:34.079+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:34.224 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:34 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:34.079+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.905348+0000 front 2024-03-21T13:29:02.905252+0000 (oldest deadline 2024-03-21T13:29:28.205380+0000) 2024-03-21T13:29:34.707 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:34 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:34.223+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:34.707 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:34 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:34.223+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.681769+0000 front 2024-03-21T13:29:02.681923+0000 (oldest deadline 2024-03-21T13:29:27.981820+0000) 2024-03-21T13:29:35.457 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:34 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:34.966+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:56.549589+0000 front 2024-03-21T13:28:56.549461+0000 (oldest deadline 2024-03-21T13:29:20.649493+0000) 2024-03-21T13:29:35.457 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:34 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:34.966+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:04.750466+0000 front 2024-03-21T13:29:04.750495+0000 (oldest deadline 2024-03-21T13:29:27.650390+0000) 2024-03-21T13:29:35.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:35 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:35.190+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:35.458 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:35 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:35.190+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.681769+0000 front 2024-03-21T13:29:02.681923+0000 (oldest deadline 2024-03-21T13:29:27.981820+0000) 2024-03-21T13:29:35.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:35 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:35.077+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:35.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:35 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:35.077+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.905348+0000 front 2024-03-21T13:29:02.905252+0000 (oldest deadline 2024-03-21T13:29:28.205380+0000) 2024-03-21T13:29:35.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:35 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:35.077+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.306409+0000 front 2024-03-21T13:29:12.306355+0000 (oldest deadline 2024-03-21T13:29:34.606303+0000) 2024-03-21T13:29:36.207 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:35 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:35.948+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:56.549589+0000 front 2024-03-21T13:28:56.549461+0000 (oldest deadline 2024-03-21T13:29:20.649493+0000) 2024-03-21T13:29:36.207 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:35 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:35.948+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:04.750466+0000 front 2024-03-21T13:29:04.750495+0000 (oldest deadline 2024-03-21T13:29:27.650390+0000) 2024-03-21T13:29:36.207 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:36 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:36.148+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:36.207 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:36 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:36.148+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.681769+0000 front 2024-03-21T13:29:02.681923+0000 (oldest deadline 2024-03-21T13:29:27.981820+0000) 2024-03-21T13:29:36.208 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:36 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:36.030+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:36.208 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:36 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:36.030+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.905348+0000 front 2024-03-21T13:29:02.905252+0000 (oldest deadline 2024-03-21T13:29:28.205380+0000) 2024-03-21T13:29:36.208 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:36 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:36.030+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.306409+0000 front 2024-03-21T13:29:12.306355+0000 (oldest deadline 2024-03-21T13:29:34.606303+0000) 2024-03-21T13:29:37.457 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:36 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:36.957+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:56.549589+0000 front 2024-03-21T13:28:56.549461+0000 (oldest deadline 2024-03-21T13:29:20.649493+0000) 2024-03-21T13:29:37.457 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:36 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:36.957+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:04.750466+0000 front 2024-03-21T13:29:04.750495+0000 (oldest deadline 2024-03-21T13:29:27.650390+0000) 2024-03-21T13:29:37.457 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:36 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5[45439]: 2024-03-21T13:29:36.957+0000 7f43a5931700 -1 osd.5 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:10.352515+0000 front 2024-03-21T13:29:10.352537+0000 (oldest deadline 2024-03-21T13:29:36.252581+0000) 2024-03-21T13:29:37.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:37 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:37.131+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:37.458 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:37 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:37.131+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.681769+0000 front 2024-03-21T13:29:02.681923+0000 (oldest deadline 2024-03-21T13:29:27.981820+0000) 2024-03-21T13:29:37.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:37 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:37.058+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:37.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:37 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:37.058+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.905348+0000 front 2024-03-21T13:29:02.905252+0000 (oldest deadline 2024-03-21T13:29:28.205380+0000) 2024-03-21T13:29:37.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:37 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:37.058+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.306409+0000 front 2024-03-21T13:29:12.306355+0000 (oldest deadline 2024-03-21T13:29:34.606303+0000) 2024-03-21T13:29:38.047 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:37 smithi155 podman[80288]: 2024-03-21 13:29:37.785091381 +0000 UTC m=+5.612564086 container died f59e240df64d3bd9f3fa1e984c4cc319108a83d1b46718650053b65f3f7b9406 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-03-21T13:29:38.421 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:38 smithi155 podman[80288]: 2024-03-21 13:29:38.16206544 +0000 UTC m=+5.989538144 container cleanup f59e240df64d3bd9f3fa1e984c4cc319108a83d1b46718650053b65f3f7b9406 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-21T13:29:38.421 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:38 smithi155 bash[80288]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5 2024-03-21T13:29:38.421 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:38 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:38.132+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:38.421 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:38 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:38.132+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.681769+0000 front 2024-03-21T13:29:02.681923+0000 (oldest deadline 2024-03-21T13:29:27.981820+0000) 2024-03-21T13:29:38.422 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:38 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:38.047+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:38.422 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:38 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:38.047+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.905348+0000 front 2024-03-21T13:29:02.905252+0000 (oldest deadline 2024-03-21T13:29:28.205380+0000) 2024-03-21T13:29:38.422 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:38 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:38.047+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.306409+0000 front 2024-03-21T13:29:12.306355+0000 (oldest deadline 2024-03-21T13:29:34.606303+0000) 2024-03-21T13:29:38.707 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:38 smithi155 podman[80312]: 2024-03-21 13:29:38.42170607 +0000 UTC m=+0.632591360 container remove f59e240df64d3bd9f3fa1e984c4cc319108a83d1b46718650053b65f3f7b9406 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-21T13:29:39.098 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:38 smithi155 podman[80410]: 2024-03-21 13:29:38.685300033 +0000 UTC m=+0.018645943 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93 2024-03-21T13:29:39.098 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:38 smithi155 podman[80410]: 2024-03-21 13:29:38.905891433 +0000 UTC m=+0.239237342 container create 6c7fe66a295a6fa6fb1f1418ba3cad2be5c30b6be52f330a0148286a3b72a6c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, ceph=True) 2024-03-21T13:29:39.440 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:39 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:39.130+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:39.440 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:39 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:39.130+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.681769+0000 front 2024-03-21T13:29:02.681923+0000 (oldest deadline 2024-03-21T13:29:27.981820+0000) 2024-03-21T13:29:39.440 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:39 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:39.130+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.682869+0000 front 2024-03-21T13:29:12.682762+0000 (oldest deadline 2024-03-21T13:29:38.483274+0000) 2024-03-21T13:29:39.440 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:39 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:39.097+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:39.440 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:39 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:39.097+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.905348+0000 front 2024-03-21T13:29:02.905252+0000 (oldest deadline 2024-03-21T13:29:28.205380+0000) 2024-03-21T13:29:39.440 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:39 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:39.097+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.306409+0000 front 2024-03-21T13:29:12.306355+0000 (oldest deadline 2024-03-21T13:29:34.606303+0000) 2024-03-21T13:29:39.440 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:39 smithi155 podman[80410]: 2024-03-21 13:29:39.141929772 +0000 UTC m=+0.475275693 container init 6c7fe66a295a6fa6fb1f1418ba3cad2be5c30b6be52f330a0148286a3b72a6c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-21T13:29:39.441 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:39 smithi155 podman[80410]: 2024-03-21 13:29:39.148062576 +0000 UTC m=+0.481408490 container start 6c7fe66a295a6fa6fb1f1418ba3cad2be5c30b6be52f330a0148286a3b72a6c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319) 2024-03-21T13:29:39.441 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:39 smithi155 podman[80410]: 2024-03-21 13:29:39.233487202 +0000 UTC m=+0.566833126 container attach 6c7fe66a295a6fa6fb1f1418ba3cad2be5c30b6be52f330a0148286a3b72a6c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T13:29:39.707 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:39 smithi155 podman[80410]: 2024-03-21 13:29:39.440914017 +0000 UTC m=+0.774259935 container died 6c7fe66a295a6fa6fb1f1418ba3cad2be5c30b6be52f330a0148286a3b72a6c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-03-21T13:29:40.072 INFO:journalctl@ceph.osd.5.smithi155.stdout:Mar 21 13:29:40 smithi155 podman[80456]: 2024-03-21 13:29:40.070293594 +0000 UTC m=+0.619822252 container remove 6c7fe66a295a6fa6fb1f1418ba3cad2be5c30b6be52f330a0148286a3b72a6c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, ceph=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T13:29:40.090 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@osd.5.service' 2024-03-21T13:29:40.133 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T13:29:40.133 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-21T13:29:40.133 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-21T13:29:40.133 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@osd.6 2024-03-21T13:29:40.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:40 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:40.110+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:40.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:40 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:40.110+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.905348+0000 front 2024-03-21T13:29:02.905252+0000 (oldest deadline 2024-03-21T13:29:28.205380+0000) 2024-03-21T13:29:40.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:40 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:40.110+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.306409+0000 front 2024-03-21T13:29:12.306355+0000 (oldest deadline 2024-03-21T13:29:34.606303+0000) 2024-03-21T13:29:40.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:40 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:40.166+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:40.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:40 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:40.166+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.681769+0000 front 2024-03-21T13:29:02.681923+0000 (oldest deadline 2024-03-21T13:29:27.981820+0000) 2024-03-21T13:29:40.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:40 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:40.166+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.682869+0000 front 2024-03-21T13:29:12.682762+0000 (oldest deadline 2024-03-21T13:29:38.483274+0000) 2024-03-21T13:29:40.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:40 smithi155 systemd[1]: Stopping Ceph osd.6 for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T13:29:40.957 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:40 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:40.458+0000 7fe2bbcac700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T13:29:40.957 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:40 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:40.458+0000 7fe2bbcac700 -1 osd.6 838 *** Got signal Terminated *** 2024-03-21T13:29:40.957 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:40 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:40.458+0000 7fe2bbcac700 -1 osd.6 838 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T13:29:41.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:41 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:41.210+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:41.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:41 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:41.210+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.681769+0000 front 2024-03-21T13:29:02.681923+0000 (oldest deadline 2024-03-21T13:29:27.981820+0000) 2024-03-21T13:29:41.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:41 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:41.210+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.682869+0000 front 2024-03-21T13:29:12.682762+0000 (oldest deadline 2024-03-21T13:29:38.483274+0000) 2024-03-21T13:29:41.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:41 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:41.063+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:41.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:41 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:41.063+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.905348+0000 front 2024-03-21T13:29:02.905252+0000 (oldest deadline 2024-03-21T13:29:28.205380+0000) 2024-03-21T13:29:41.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:41 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:41.063+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.306409+0000 front 2024-03-21T13:29:12.306355+0000 (oldest deadline 2024-03-21T13:29:34.606303+0000) 2024-03-21T13:29:42.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:42.178+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:42.458 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:42.178+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.681769+0000 front 2024-03-21T13:29:02.681923+0000 (oldest deadline 2024-03-21T13:29:27.981820+0000) 2024-03-21T13:29:42.458 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:42.178+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.682869+0000 front 2024-03-21T13:29:12.682762+0000 (oldest deadline 2024-03-21T13:29:38.483274+0000) 2024-03-21T13:29:42.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:42.061+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:42.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:42.061+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.905348+0000 front 2024-03-21T13:29:02.905252+0000 (oldest deadline 2024-03-21T13:29:28.205380+0000) 2024-03-21T13:29:42.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:42.061+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.306409+0000 front 2024-03-21T13:29:12.306355+0000 (oldest deadline 2024-03-21T13:29:34.606303+0000) 2024-03-21T13:29:42.458 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 13:29:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T13:29:42.290Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-03-21T13:29:42.458 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 13:29:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T13:29:42.290Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-03-21T13:29:42.459 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 13:29:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T13:29:42.293Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-03-21T13:29:42.459 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 13:29:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T13:29:42.293Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-03-21T13:29:42.459 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 13:29:42 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T13:29:42.293Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.43:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.43:8765: connect: connection refused" 2024-03-21T13:29:43.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:43 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:43.159+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:43.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:43 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:43.159+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.681769+0000 front 2024-03-21T13:29:02.681923+0000 (oldest deadline 2024-03-21T13:29:27.981820+0000) 2024-03-21T13:29:43.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:43 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:43.159+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.682869+0000 front 2024-03-21T13:29:12.682762+0000 (oldest deadline 2024-03-21T13:29:38.483274+0000) 2024-03-21T13:29:43.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:43 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:43.022+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:43.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:43 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:43.022+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.905348+0000 front 2024-03-21T13:29:02.905252+0000 (oldest deadline 2024-03-21T13:29:28.205380+0000) 2024-03-21T13:29:43.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:43 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:43.022+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.306409+0000 front 2024-03-21T13:29:12.306355+0000 (oldest deadline 2024-03-21T13:29:34.606303+0000) 2024-03-21T13:29:43.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:43 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:43.022+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-03-21T13:29:18.707284+0000 front 2024-03-21T13:29:18.707269+0000 (oldest deadline 2024-03-21T13:29:42.807191+0000) 2024-03-21T13:29:43.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:43 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:43.022+0000 7fe820c1f700 -1 osd.7 838 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14550.0:2241 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e838) 2024-03-21T13:29:44.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:44 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:44.190+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:44.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:44 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:44.190+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.681769+0000 front 2024-03-21T13:29:02.681923+0000 (oldest deadline 2024-03-21T13:29:27.981820+0000) 2024-03-21T13:29:44.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:44 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:44.190+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.682869+0000 front 2024-03-21T13:29:12.682762+0000 (oldest deadline 2024-03-21T13:29:38.483274+0000) 2024-03-21T13:29:44.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:44 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:44.190+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-03-21T13:29:18.483888+0000 front 2024-03-21T13:29:18.483921+0000 (oldest deadline 2024-03-21T13:29:43.183817+0000) 2024-03-21T13:29:44.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:44 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:44.065+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:44.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:44 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:44.065+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.905348+0000 front 2024-03-21T13:29:02.905252+0000 (oldest deadline 2024-03-21T13:29:28.205380+0000) 2024-03-21T13:29:44.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:44 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:44.065+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.306409+0000 front 2024-03-21T13:29:12.306355+0000 (oldest deadline 2024-03-21T13:29:34.606303+0000) 2024-03-21T13:29:44.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:44 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:44.065+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-03-21T13:29:18.707284+0000 front 2024-03-21T13:29:18.707269+0000 (oldest deadline 2024-03-21T13:29:42.807191+0000) 2024-03-21T13:29:44.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:44 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:44.065+0000 7fe820c1f700 -1 osd.7 838 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14550.0:2241 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e838) 2024-03-21T13:29:45.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:45 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:45.179+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:53.280891+0000 front 2024-03-21T13:28:53.280708+0000 (oldest deadline 2024-03-21T13:29:18.580647+0000) 2024-03-21T13:29:45.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:45 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:45.179+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.681769+0000 front 2024-03-21T13:29:02.681923+0000 (oldest deadline 2024-03-21T13:29:27.981820+0000) 2024-03-21T13:29:45.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:45 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:45.179+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.682869+0000 front 2024-03-21T13:29:12.682762+0000 (oldest deadline 2024-03-21T13:29:38.483274+0000) 2024-03-21T13:29:45.457 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:45 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6[51406]: 2024-03-21T13:29:45.179+0000 7fe2b82a0700 -1 osd.6 838 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-03-21T13:29:18.483888+0000 front 2024-03-21T13:29:18.483921+0000 (oldest deadline 2024-03-21T13:29:43.183817+0000) 2024-03-21T13:29:45.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:45 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:45.018+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:45.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:45 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:45.018+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.905348+0000 front 2024-03-21T13:29:02.905252+0000 (oldest deadline 2024-03-21T13:29:28.205380+0000) 2024-03-21T13:29:45.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:45 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:45.018+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.306409+0000 front 2024-03-21T13:29:12.306355+0000 (oldest deadline 2024-03-21T13:29:34.606303+0000) 2024-03-21T13:29:45.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:45 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:45.018+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-03-21T13:29:18.707284+0000 front 2024-03-21T13:29:18.707269+0000 (oldest deadline 2024-03-21T13:29:42.807191+0000) 2024-03-21T13:29:45.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:45 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:45.018+0000 7fe820c1f700 -1 osd.7 838 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14550.0:2241 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e838) 2024-03-21T13:29:46.115 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:45 smithi155 podman[80540]: 2024-03-21 13:29:45.8086594 +0000 UTC m=+5.451040749 container died 0a6fc204244cd1025e80121c5601a3b875865993ec339080a804a127b98e8b41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-03-21T13:29:46.115 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:46 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:46.048+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:46.115 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:46 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:46.048+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.905348+0000 front 2024-03-21T13:29:02.905252+0000 (oldest deadline 2024-03-21T13:29:28.205380+0000) 2024-03-21T13:29:46.115 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:46 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:46.048+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.306409+0000 front 2024-03-21T13:29:12.306355+0000 (oldest deadline 2024-03-21T13:29:34.606303+0000) 2024-03-21T13:29:46.115 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:46 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:46.048+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-03-21T13:29:18.707284+0000 front 2024-03-21T13:29:18.707269+0000 (oldest deadline 2024-03-21T13:29:42.807191+0000) 2024-03-21T13:29:46.115 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:46 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:46.048+0000 7fe820c1f700 -1 osd.7 838 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14550.0:2241 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e838) 2024-03-21T13:29:46.440 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:46 smithi155 podman[80540]: 2024-03-21 13:29:46.22284852 +0000 UTC m=+5.865229912 container cleanup 0a6fc204244cd1025e80121c5601a3b875865993ec339080a804a127b98e8b41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-21T13:29:46.440 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:46 smithi155 bash[80540]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6 2024-03-21T13:29:46.707 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:46 smithi155 podman[80562]: 2024-03-21 13:29:46.440593498 +0000 UTC m=+0.630529488 container remove 0a6fc204244cd1025e80121c5601a3b875865993ec339080a804a127b98e8b41 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319) 2024-03-21T13:29:47.018 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:46 smithi155 podman[80657]: 2024-03-21 13:29:46.758036412 +0000 UTC m=+0.114366236 container create ba2fa7534ac5aaae662e875c804df959785276e81c9f33f046cb44b9a6a7417a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T13:29:47.018 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:46 smithi155 podman[80657]: 2024-03-21 13:29:46.66136031 +0000 UTC m=+0.017690164 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93 2024-03-21T13:29:47.018 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:46 smithi155 podman[80657]: 2024-03-21 13:29:46.935283325 +0000 UTC m=+0.291613170 container init ba2fa7534ac5aaae662e875c804df959785276e81c9f33f046cb44b9a6a7417a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6-deactivate, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=) 2024-03-21T13:29:47.018 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:46 smithi155 podman[80657]: 2024-03-21 13:29:46.940858037 +0000 UTC m=+0.297187871 container start ba2fa7534ac5aaae662e875c804df959785276e81c9f33f046cb44b9a6a7417a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-03-21T13:29:47.018 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:47 smithi155 podman[80657]: 2024-03-21 13:29:47.018389622 +0000 UTC m=+0.374719457 container attach ba2fa7534ac5aaae662e875c804df959785276e81c9f33f046cb44b9a6a7417a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-21T13:29:47.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:47 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:47.097+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:47.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:47 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:47.097+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.905348+0000 front 2024-03-21T13:29:02.905252+0000 (oldest deadline 2024-03-21T13:29:28.205380+0000) 2024-03-21T13:29:47.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:47 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:47.097+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.306409+0000 front 2024-03-21T13:29:12.306355+0000 (oldest deadline 2024-03-21T13:29:34.606303+0000) 2024-03-21T13:29:47.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:47 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:47.097+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-03-21T13:29:18.707284+0000 front 2024-03-21T13:29:18.707269+0000 (oldest deadline 2024-03-21T13:29:42.807191+0000) 2024-03-21T13:29:47.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:47 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:47.097+0000 7fe820c1f700 -1 osd.7 838 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14550.0:2241 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e838) 2024-03-21T13:29:47.458 INFO:journalctl@ceph.osd.6.smithi155.stdout:Mar 21 13:29:47 smithi155 podman[80657]: 2024-03-21 13:29:47.229884841 +0000 UTC m=+0.586214674 container died ba2fa7534ac5aaae662e875c804df959785276e81c9f33f046cb44b9a6a7417a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-6-deactivate, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-03-21T13:29:47.824 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@osd.6.service' 2024-03-21T13:29:47.865 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T13:29:47.865 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-21T13:29:47.865 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-21T13:29:47.865 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@osd.7 2024-03-21T13:29:48.108 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:47 smithi155 systemd[1]: Stopping Ceph osd.7 for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T13:29:48.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:48 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:48.107+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:48.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:48 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:48.107+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.905348+0000 front 2024-03-21T13:29:02.905252+0000 (oldest deadline 2024-03-21T13:29:28.205380+0000) 2024-03-21T13:29:48.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:48 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:48.107+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.306409+0000 front 2024-03-21T13:29:12.306355+0000 (oldest deadline 2024-03-21T13:29:34.606303+0000) 2024-03-21T13:29:48.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:48 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:48.107+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-03-21T13:29:18.707284+0000 front 2024-03-21T13:29:18.707269+0000 (oldest deadline 2024-03-21T13:29:42.807191+0000) 2024-03-21T13:29:48.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:48 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:48.108+0000 7fe820c1f700 -1 osd.7 838 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14550.0:2241 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e838) 2024-03-21T13:29:48.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:48 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:48.209+0000 7fe82462b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T13:29:48.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:48 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:48.209+0000 7fe82462b700 -1 osd.7 838 *** Got signal Terminated *** 2024-03-21T13:29:48.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:48 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:48.209+0000 7fe82462b700 -1 osd.7 838 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T13:29:49.358 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:49.098+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:49.358 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:49.098+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.905348+0000 front 2024-03-21T13:29:02.905252+0000 (oldest deadline 2024-03-21T13:29:28.205380+0000) 2024-03-21T13:29:49.358 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:49.098+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.306409+0000 front 2024-03-21T13:29:12.306355+0000 (oldest deadline 2024-03-21T13:29:34.606303+0000) 2024-03-21T13:29:49.358 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:49.098+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-03-21T13:29:18.707284+0000 front 2024-03-21T13:29:18.707269+0000 (oldest deadline 2024-03-21T13:29:42.807191+0000) 2024-03-21T13:29:49.358 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:49 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:49.098+0000 7fe820c1f700 -1 osd.7 838 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14550.0:2241 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e838) 2024-03-21T13:29:50.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:50 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:50.123+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:50.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:50 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:50.123+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.905348+0000 front 2024-03-21T13:29:02.905252+0000 (oldest deadline 2024-03-21T13:29:28.205380+0000) 2024-03-21T13:29:50.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:50 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:50.123+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.306409+0000 front 2024-03-21T13:29:12.306355+0000 (oldest deadline 2024-03-21T13:29:34.606303+0000) 2024-03-21T13:29:50.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:50 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:50.123+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-03-21T13:29:18.707284+0000 front 2024-03-21T13:29:18.707269+0000 (oldest deadline 2024-03-21T13:29:42.807191+0000) 2024-03-21T13:29:50.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:50 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:50.124+0000 7fe820c1f700 -1 osd.7 838 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14550.0:2241 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e838) 2024-03-21T13:29:51.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:51 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:51.111+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:51.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:51 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:51.111+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.905348+0000 front 2024-03-21T13:29:02.905252+0000 (oldest deadline 2024-03-21T13:29:28.205380+0000) 2024-03-21T13:29:51.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:51 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:51.111+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.306409+0000 front 2024-03-21T13:29:12.306355+0000 (oldest deadline 2024-03-21T13:29:34.606303+0000) 2024-03-21T13:29:51.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:51 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:51.111+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-03-21T13:29:18.707284+0000 front 2024-03-21T13:29:18.707269+0000 (oldest deadline 2024-03-21T13:29:42.807191+0000) 2024-03-21T13:29:51.458 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:51 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:51.111+0000 7fe820c1f700 -1 osd.7 838 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14550.0:2241 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e838) 2024-03-21T13:29:52.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:52 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:52.079+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:52.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:52 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:52.079+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.905348+0000 front 2024-03-21T13:29:02.905252+0000 (oldest deadline 2024-03-21T13:29:28.205380+0000) 2024-03-21T13:29:52.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:52 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:52.079+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.306409+0000 front 2024-03-21T13:29:12.306355+0000 (oldest deadline 2024-03-21T13:29:34.606303+0000) 2024-03-21T13:29:52.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:52 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:52.079+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-03-21T13:29:18.707284+0000 front 2024-03-21T13:29:18.707269+0000 (oldest deadline 2024-03-21T13:29:42.807191+0000) 2024-03-21T13:29:52.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:52 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:52.079+0000 7fe820c1f700 -1 osd.7 838 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14550.0:2241 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e838) 2024-03-21T13:29:53.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:53 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:53.064+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6803 osd.0 since back 2024-03-21T13:28:57.804053+0000 front 2024-03-21T13:28:57.804036+0000 (oldest deadline 2024-03-21T13:29:18.303808+0000) 2024-03-21T13:29:53.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:53 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:53.064+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6807 osd.1 since back 2024-03-21T13:29:02.905348+0000 front 2024-03-21T13:29:02.905252+0000 (oldest deadline 2024-03-21T13:29:28.205380+0000) 2024-03-21T13:29:53.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:53 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:53.064+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6811 osd.2 since back 2024-03-21T13:29:12.306409+0000 front 2024-03-21T13:29:12.306355+0000 (oldest deadline 2024-03-21T13:29:34.606303+0000) 2024-03-21T13:29:53.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:53 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:53.064+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.43:6815 osd.3 since back 2024-03-21T13:29:18.707284+0000 front 2024-03-21T13:29:18.707269+0000 (oldest deadline 2024-03-21T13:29:42.807191+0000) 2024-03-21T13:29:53.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:53 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:53.064+0000 7fe820c1f700 -1 osd.7 838 heartbeat_check: no reply from 172.21.15.155:6802 osd.4 since back 2024-03-21T13:29:28.008557+0000 front 2024-03-21T13:29:28.008668+0000 (oldest deadline 2024-03-21T13:29:52.708618+0000) 2024-03-21T13:29:53.457 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:53 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7[57864]: 2024-03-21T13:29:53.064+0000 7fe820c1f700 -1 osd.7 838 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14550.0:2241 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e838) 2024-03-21T13:29:53.957 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:53 smithi155 podman[80791]: 2024-03-21 13:29:53.687996905 +0000 UTC m=+5.603736229 container died e1fe70dacb17bb45ef3a018bc761a931d6cf8249a825b3a429daaf65095aa13d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-21T13:29:54.308 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:54 smithi155 podman[80791]: 2024-03-21 13:29:54.076006915 +0000 UTC m=+5.991746239 container cleanup e1fe70dacb17bb45ef3a018bc761a931d6cf8249a825b3a429daaf65095aa13d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-03-21T13:29:54.308 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:54 smithi155 bash[80791]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7 2024-03-21T13:29:54.634 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:54 smithi155 podman[80825]: 2024-03-21 13:29:54.309096306 +0000 UTC m=+0.614704094 container remove e1fe70dacb17bb45ef3a018bc761a931d6cf8249a825b3a429daaf65095aa13d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-21T13:29:54.937 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:54 smithi155 podman[80920]: 2024-03-21 13:29:54.634793066 +0000 UTC m=+0.111253191 container create 6d39d738906fd2b24b6ef687a66a2d7faa82746e69ad021edbb4e1b4934923f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319) 2024-03-21T13:29:54.937 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:54 smithi155 podman[80920]: 2024-03-21 13:29:54.540935199 +0000 UTC m=+0.017395334 image pull 19e12293512ff7978c8adb5f72a829060dd47f07bfd6903b6b3178d2ebc29e4e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93 2024-03-21T13:29:54.937 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:54 smithi155 podman[80920]: 2024-03-21 13:29:54.845685613 +0000 UTC m=+0.322145747 container init 6d39d738906fd2b24b6ef687a66a2d7faa82746e69ad021edbb4e1b4934923f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T13:29:54.937 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:54 smithi155 podman[80920]: 2024-03-21 13:29:54.851530768 +0000 UTC m=+0.327990894 container start 6d39d738906fd2b24b6ef687a66a2d7faa82746e69ad021edbb4e1b4934923f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-03-21T13:29:54.937 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:54 smithi155 podman[80920]: 2024-03-21 13:29:54.937151556 +0000 UTC m=+0.413611693 container attach 6d39d738906fd2b24b6ef687a66a2d7faa82746e69ad021edbb4e1b4934923f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe) 2024-03-21T13:29:55.207 INFO:journalctl@ceph.osd.7.smithi155.stdout:Mar 21 13:29:55 smithi155 podman[80920]: 2024-03-21 13:29:55.138068762 +0000 UTC m=+0.614528888 container died 6d39d738906fd2b24b6ef687a66a2d7faa82746e69ad021edbb4e1b4934923f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe) 2024-03-21T13:29:55.704 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@osd.7.service' 2024-03-21T13:29:55.744 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T13:29:55.744 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-21T13:29:55.744 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-03-21T13:29:55.745 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@rgw.foo.a 2024-03-21T13:29:56.034 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Mar 21 13:29:55 smithi043 systemd[1]: Stopping Ceph rgw.foo.a for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T13:29:56.287 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Mar 21 13:29:56 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-rgw-foo-a[69848]: 2024-03-21T13:29:56.061+0000 7f36322da700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T13:29:56.287 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Mar 21 13:29:56 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-rgw-foo-a[69848]: 2024-03-21T13:29:56.061+0000 7f3640b5fac0 -1 shutting down 2024-03-21T13:30:06.359 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Mar 21 13:30:06 smithi043 bash[161526]: time="2024-03-21T13:30:06Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-03-21T13:30:06.359 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Mar 21 13:30:06 smithi043 podman[161526]: 2024-03-21 13:30:06.084460305 +0000 UTC m=+10.145825564 container died 04f8420e74d02a4ad112aaf4ffe81e89f06b468007a81263a56603e642a04886 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-rgw-foo-a, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-03-21T13:30:06.723 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Mar 21 13:30:06 smithi043 podman[161526]: 2024-03-21 13:30:06.458286052 +0000 UTC m=+10.519651345 container cleanup 04f8420e74d02a4ad112aaf4ffe81e89f06b468007a81263a56603e642a04886 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-rgw-foo-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True) 2024-03-21T13:30:06.723 INFO:journalctl@ceph.rgw.foo.a.smithi043.stdout:Mar 21 13:30:06 smithi043 bash[161526]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-rgw-foo-a 2024-03-21T13:30:06.884 DEBUG:teuthology.orchestra.run.smithi043:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@rgw.foo.a.service' 2024-03-21T13:30:06.923 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T13:30:06.924 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-03-21T13:30:06.924 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-03-21T13:30:06.924 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@prometheus.a 2024-03-21T13:30:07.207 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 13:30:06 smithi155 systemd[1]: Stopping Ceph prometheus.a for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T13:30:07.542 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 13:30:07 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T13:30:07.244Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-03-21T13:30:07.543 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 13:30:07 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T13:30:07.245Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-03-21T13:30:07.543 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 13:30:07 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T13:30:07.245Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-03-21T13:30:07.543 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 13:30:07 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T13:30:07.245Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-03-21T13:30:07.543 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 13:30:07 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T13:30:07.245Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-03-21T13:30:07.543 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 13:30:07 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T13:30:07.245Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-03-21T13:30:07.543 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 13:30:07 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T13:30:07.245Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-03-21T13:30:07.543 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 13:30:07 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T13:30:07.245Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-03-21T13:30:07.543 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 13:30:07 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T13:30:07.245Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-03-21T13:30:07.543 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 13:30:07 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T13:30:07.246Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-03-21T13:30:07.543 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 13:30:07 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T13:30:07.246Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-03-21T13:30:07.543 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 13:30:07 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a[68885]: ts=2024-03-21T13:30:07.246Z caller=main.go:1147 level=info msg="See you next time!" 2024-03-21T13:30:07.543 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 13:30:07 smithi155 podman[81051]: 2024-03-21 13:30:07.255288145 +0000 UTC m=+0.132098773 container died a2db772e1052bf870709ffcaf6442b7fbd95c0f0a78ebfdbd121732889047fb4 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-21T13:30:07.884 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 13:30:07 smithi155 podman[81051]: 2024-03-21 13:30:07.650592985 +0000 UTC m=+0.527403612 container cleanup a2db772e1052bf870709ffcaf6442b7fbd95c0f0a78ebfdbd121732889047fb4 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-03-21T13:30:07.884 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Mar 21 13:30:07 smithi155 bash[81051]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-prometheus-a 2024-03-21T13:30:08.007 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@prometheus.a.service' 2024-03-21T13:30:08.043 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T13:30:08.043 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-03-21T13:30:08.043 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm rm-cluster --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-21T13:30:08.239 INFO:teuthology.orchestra.run.smithi043.stdout:Deleting cluster with fsid: c06c8d1a-e780-11ee-95cd-87774f69a715 2024-03-21T13:30:09.984 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 13:30:09 smithi043 systemd[1]: Stopping Ceph alertmanager.a for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T13:30:10.372 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 13:30:10 smithi043 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a[80851]: ts=2024-03-21T13:30:10.073Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-03-21T13:30:10.372 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 13:30:10 smithi043 podman[162048]: 2024-03-21 13:30:10.115115829 +0000 UTC m=+0.156807438 container died 1687430f4e4f00222d1dba9d45cdfa5f19df49c7ebb57625bbbb2da9313ad43a (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-21T13:30:10.733 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 13:30:10 smithi043 podman[162048]: 2024-03-21 13:30:10.58824021 +0000 UTC m=+0.629931827 container cleanup 1687430f4e4f00222d1dba9d45cdfa5f19df49c7ebb57625bbbb2da9313ad43a (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-21T13:30:10.733 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 13:30:10 smithi043 bash[162048]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a 2024-03-21T13:30:11.134 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 13:30:10 smithi043 podman[162065]: 2024-03-21 13:30:10.839378019 +0000 UTC m=+0.722057902 container remove 1687430f4e4f00222d1dba9d45cdfa5f19df49c7ebb57625bbbb2da9313ad43a (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-03-21T13:30:11.134 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 13:30:10 smithi043 podman[162065]: 2024-03-21 13:30:10.914891592 +0000 UTC m=+0.797571478 volume remove 6d396cca714dfe6c9bf7bee0c28d0c194b9bdca4e6e936c5e77ffda2e34c5a54 2024-03-21T13:30:11.134 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 13:30:10 smithi043 systemd[1]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-03-21T13:30:11.134 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 13:30:10 smithi043 systemd[1]: Stopped Ceph alertmanager.a for c06c8d1a-e780-11ee-95cd-87774f69a715. 2024-03-21T13:30:11.134 INFO:journalctl@ceph.alertmanager.a.smithi043.stdout:Mar 21 13:30:10 smithi043 systemd[1]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@alertmanager.a.service: Consumed 2.442s CPU time. 2024-03-21T13:30:11.397 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 13:30:11 smithi043 systemd[1]: Stopping Ceph node-exporter.a for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T13:30:11.701 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 13:30:11 smithi043 podman[162182]: 2024-03-21 13:30:11.397933554 +0000 UTC m=+0.125879004 container died 314670e24b1af1ab3f986d4913c9318da0945d2f062404fc5d19f1cf64bf471b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-21T13:30:11.983 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 13:30:11 smithi043 podman[162182]: 2024-03-21 13:30:11.800734797 +0000 UTC m=+0.528680239 container cleanup 314670e24b1af1ab3f986d4913c9318da0945d2f062404fc5d19f1cf64bf471b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-21T13:30:11.983 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 13:30:11 smithi043 bash[162182]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a 2024-03-21T13:30:12.264 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 13:30:11 smithi043 podman[162201]: 2024-03-21 13:30:11.997726972 +0000 UTC m=+0.600312840 container remove 314670e24b1af1ab3f986d4913c9318da0945d2f062404fc5d19f1cf64bf471b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-03-21T13:30:12.264 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 13:30:12 smithi043 systemd[1]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-03-21T13:30:12.265 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 13:30:12 smithi043 systemd[1]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-03-21T13:30:12.265 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 13:30:12 smithi043 systemd[1]: Stopped Ceph node-exporter.a for c06c8d1a-e780-11ee-95cd-87774f69a715. 2024-03-21T13:30:12.265 INFO:journalctl@ceph.node-exporter.a.smithi043.stdout:Mar 21 13:30:12 smithi043 systemd[1]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@node-exporter.a.service: Consumed 5.392s CPU time. 2024-03-21T13:30:12.813 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm rm-cluster --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-21T13:30:13.005 INFO:teuthology.orchestra.run.smithi155.stdout:Deleting cluster with fsid: c06c8d1a-e780-11ee-95cd-87774f69a715 2024-03-21T13:30:14.457 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:30:14 smithi155 systemd[1]: Stopping Ceph iscsi.iscsi.a for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T13:30:14.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:30:14 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a[63367]: debug Shutdown received 2024-03-21T13:30:24.806 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:30:24 smithi155 bash[81472]: time="2024-03-21T13:30:24Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-03-21T13:30:24.806 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:30:24 smithi155 podman[81472]: 2024-03-21 13:30:24.514916788 +0000 UTC m=+10.133640635 container died f6ef43e8976ade7ff34199a453bcd937009acd06b7b927a09eb5c736aaf67533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-03-21T13:30:25.128 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:30:24 smithi155 podman[81472]: 2024-03-21 13:30:24.903272384 +0000 UTC m=+10.521996223 container cleanup f6ef43e8976ade7ff34199a453bcd937009acd06b7b927a09eb5c736aaf67533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, ceph=True, CEPH_POINT_RELEASE=) 2024-03-21T13:30:25.129 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:30:24 smithi155 bash[81472]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a 2024-03-21T13:30:25.439 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:30:25 smithi155 podman[81497]: 2024-03-21 13:30:25.129383845 +0000 UTC m=+0.611980551 container remove f6ef43e8976ade7ff34199a453bcd937009acd06b7b927a09eb5c736aaf67533 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:35f4b82cbae750007a45d4eba3ff9f653c457c283782141ef353684823d81e93, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-iscsi-iscsi-a, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-guits-testing-2024-03-21-0818-960fbbe, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-03-21T13:30:25.439 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:30:25 smithi155 systemd[1]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-03-21T13:30:25.439 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:30:25 smithi155 systemd[1]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-03-21T13:30:25.439 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:30:25 smithi155 systemd[1]: Stopped Ceph iscsi.iscsi.a for c06c8d1a-e780-11ee-95cd-87774f69a715. 2024-03-21T13:30:25.439 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Mar 21 13:30:25 smithi155 systemd[1]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@iscsi.iscsi.a.service: Consumed 4.435s CPU time. 2024-03-21T13:30:26.957 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 13:30:26 smithi155 systemd[1]: Stopping Ceph grafana.a for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T13:30:27.368 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 13:30:27 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=server t=2024-03-21T13:30:27.105615801Z level=info msg="Shutdown started" reason="System signal: terminated" 2024-03-21T13:30:27.368 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 13:30:27 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=tracing t=2024-03-21T13:30:27.106020502Z level=info msg="Closing tracing" 2024-03-21T13:30:27.368 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 13:30:27 smithi155 ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a[67385]: logger=ticker t=2024-03-21T13:30:27.106011171Z level=info msg=stopped last_tick=2024-03-21T13:30:20Z 2024-03-21T13:30:27.368 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 13:30:27 smithi155 podman[81782]: 2024-03-21 13:30:27.116141027 +0000 UTC m=+0.125632323 container died 2bd2d10258083709ad040a15e28c2aa77834b114275b013ecfdc9f429a1c2fb6 (image=quay.io/ceph/grafana:9.4.12, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a, maintainer=Grafana Labs ) 2024-03-21T13:30:27.368 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 13:30:27 smithi155 podman[81782]: 2024-03-21 13:30:27.368714931 +0000 UTC m=+0.378206219 container cleanup 2bd2d10258083709ad040a15e28c2aa77834b114275b013ecfdc9f429a1c2fb6 (image=quay.io/ceph/grafana:9.4.12, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a, maintainer=Grafana Labs ) 2024-03-21T13:30:27.627 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 13:30:27 smithi155 bash[81782]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a 2024-03-21T13:30:27.957 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 13:30:27 smithi155 podman[81800]: 2024-03-21 13:30:27.628243854 +0000 UTC m=+0.512867983 container remove 2bd2d10258083709ad040a15e28c2aa77834b114275b013ecfdc9f429a1c2fb6 (image=quay.io/ceph/grafana:9.4.12, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-grafana-a, maintainer=Grafana Labs ) 2024-03-21T13:30:27.957 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 13:30:27 smithi155 systemd[1]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@grafana.a.service: Deactivated successfully. 2024-03-21T13:30:27.957 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 13:30:27 smithi155 systemd[1]: Stopped Ceph grafana.a for c06c8d1a-e780-11ee-95cd-87774f69a715. 2024-03-21T13:30:27.957 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Mar 21 13:30:27 smithi155 systemd[1]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@grafana.a.service: Consumed 24.835s CPU time. 2024-03-21T13:30:28.457 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 13:30:27 smithi155 systemd[1]: Stopping Ceph node-exporter.b for c06c8d1a-e780-11ee-95cd-87774f69a715... 2024-03-21T13:30:28.457 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 13:30:28 smithi155 podman[81918]: 2024-03-21 13:30:28.236166759 +0000 UTC m=+0.106147948 container died 390703071fd0d5186f2107ca75772f2722883635a64b59d187b35e9df5f6ca17 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-21T13:30:28.748 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 13:30:28 smithi155 podman[81918]: 2024-03-21 13:30:28.555998896 +0000 UTC m=+0.425980081 container cleanup 390703071fd0d5186f2107ca75772f2722883635a64b59d187b35e9df5f6ca17 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-21T13:30:28.748 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 13:30:28 smithi155 bash[81918]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b 2024-03-21T13:30:29.104 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 13:30:28 smithi155 podman[81937]: 2024-03-21 13:30:28.748603722 +0000 UTC m=+0.517125130 container remove 390703071fd0d5186f2107ca75772f2722883635a64b59d187b35e9df5f6ca17 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-c06c8d1a-e780-11ee-95cd-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-03-21T13:30:29.104 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 13:30:28 smithi155 systemd[1]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-03-21T13:30:29.104 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 13:30:28 smithi155 systemd[1]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-03-21T13:30:29.104 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 13:30:28 smithi155 systemd[1]: Stopped Ceph node-exporter.b for c06c8d1a-e780-11ee-95cd-87774f69a715. 2024-03-21T13:30:29.104 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Mar 21 13:30:28 smithi155 systemd[1]: ceph-c06c8d1a-e780-11ee-95cd-87774f69a715@node-exporter.b.service: Consumed 5.252s CPU time. 2024-03-21T13:30:29.599 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T13:30:29.635 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T13:30:29.663 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-21T13:30:29.665 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614946/remote/smithi043/crash 2024-03-21T13:30:29.666 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/crash -- . 2024-03-21T13:30:29.721 INFO:teuthology.orchestra.run.smithi043.stderr:tar: /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-21T13:30:29.721 INFO:teuthology.orchestra.run.smithi043.stderr:tar: Error is not recoverable: exiting now 2024-03-21T13:30:29.723 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614946/remote/smithi155/crash 2024-03-21T13:30:29.724 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/crash -- . 2024-03-21T13:30:29.757 INFO:teuthology.orchestra.run.smithi155.stderr:tar: /var/lib/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-21T13:30:29.758 INFO:teuthology.orchestra.run.smithi155.stderr:tar: Error is not recoverable: exiting now 2024-03-21T13:30:29.759 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-21T13:30:29.759 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-21T13:30:29.795 INFO:teuthology.orchestra.run.smithi043.stdout:2024-03-21T12:54:53.876180+0000 mon.a (mon.0) 1052 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-21T13:30:29.795 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-21T13:30:29.795 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[SEC\]' /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-21T13:30:29.867 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[ERR\]' /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-21T13:30:29.943 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[WRN\]' /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-21T13:30:30.016 INFO:teuthology.orchestra.run.smithi043.stdout:2024-03-21T12:54:53.876180+0000 mon.a (mon.0) 1052 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-21T13:30:30.016 INFO:tasks.cephadm:Compressing logs... 2024-03-21T13:30:30.016 DEBUG:teuthology.orchestra.run.smithi043:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-21T13:30:30.059 DEBUG:teuthology.orchestra.run.smithi155:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-21T13:30:30.083 INFO:teuthology.orchestra.run.smithi043.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-mon.a.log 2024-03-21T13:30:30.083 INFO:teuthology.orchestra.run.smithi043.stderr:: No such file or directory 2024-03-21T13:30:30.083 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-21T13:30:30.083 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-client.rgw.foo.a.log 2024-03-21T13:30:30.083 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-mon.a.log: /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-osd.1.log 2024-03-21T13:30:30.084 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-osd.3.log 2024-03-21T13:30:30.085 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-mgr.y.log 2024-03-21T13:30:30.085 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-osd.0.log 2024-03-21T13:30:30.085 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-21T13:30:30.086 INFO:teuthology.orchestra.run.smithi155.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-osd.6.log 2024-03-21T13:30:30.086 INFO:teuthology.orchestra.run.smithi155.stderr:: No such file or directory 2024-03-21T13:30:30.086 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-21T13:30:30.086 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-osd.4.log 2024-03-21T13:30:30.091 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-osd.7.log 2024-03-21T13:30:30.091 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-osd.5.log 2024-03-21T13:30:30.091 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-mgr.x.log 2024-03-21T13:30:30.091 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-osd.7.log: /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/tcmu-runner.log 2024-03-21T13:30:30.091 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-21T13:30:30.091 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/tcmu-runner.log: gzip 68.0% -- replaced with /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/tcmu-runner.log.gz 2024-03-21T13:30:30.091 INFO:teuthology.orchestra.run.smithi155.stderr: -5 --verbose -- /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph.log 2024-03-21T13:30:30.092 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-21T13:30:30.092 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph.cephadm.log: /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph.log: 81.4% -- replaced with /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-21T13:30:30.092 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-mon.b.log 2024-03-21T13:30:30.096 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-volume.log: 88.1% -- replaced with /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-21T13:30:30.096 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-21T13:30:30.097 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph.log 2024-03-21T13:30:30.097 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph.cephadm.log: 89.0% -- replaced with /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-21T13:30:30.099 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-mon.b.log: 92.4% -- replaced with /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-21T13:30:30.100 INFO:teuthology.orchestra.run.smithi155.stderr: 92.4% -- replaced with /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-mgr.x.log.gz 2024-03-21T13:30:30.105 INFO:teuthology.orchestra.run.smithi155.stderr: 90.8% -- replaced with /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-21T13:30:30.106 INFO:teuthology.orchestra.run.smithi043.stderr: 95.3% -- replaced with /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-21T13:30:30.107 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-osd.2.log 2024-03-21T13:30:30.117 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph.log: 93.4%gzip -5 --verbose -- /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-21T13:30:30.119 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/cephadm.log: 91.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-21T13:30:30.127 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-mon.c.log 2024-03-21T13:30:30.129 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-volume.log: -- replaced with /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-21T13:30:30.143 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-21T13:30:30.167 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-mon.c.log: /var/log/ceph/cephadm.log: 92.3% 90.9% -- replaced with /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-21T13:30:30.170 INFO:teuthology.orchestra.run.smithi043.stderr: 93.5% -- replaced with /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-client.rgw.foo.a.log.gz 2024-03-21T13:30:30.173 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-21T13:30:30.655 INFO:teuthology.orchestra.run.smithi043.stderr: 90.2% -- replaced with /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-mgr.y.log.gz 2024-03-21T13:30:32.020 INFO:teuthology.orchestra.run.smithi155.stderr: 92.5% -- replaced with /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-mon.b.log.gz 2024-03-21T13:30:32.302 INFO:teuthology.orchestra.run.smithi043.stderr: 92.1% -- replaced with /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-mon.c.log.gz 2024-03-21T13:30:34.024 INFO:teuthology.orchestra.run.smithi043.stderr: 91.8% -- replaced with /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-mon.a.log.gz 2024-03-21T13:30:39.194 INFO:teuthology.orchestra.run.smithi155.stderr: 93.9% -- replaced with /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-osd.6.log.gz 2024-03-21T13:30:39.373 INFO:teuthology.orchestra.run.smithi155.stderr: 94.0% -- replaced with /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-osd.5.log.gz 2024-03-21T13:30:39.400 INFO:teuthology.orchestra.run.smithi155.stderr: 94.0% -- replaced with /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-osd.4.log.gz 2024-03-21T13:30:39.436 INFO:teuthology.orchestra.run.smithi155.stderr: 93.9% -- replaced with /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-osd.7.log.gz 2024-03-21T13:30:39.438 INFO:teuthology.orchestra.run.smithi155.stderr: 2024-03-21T13:30:39.438 INFO:teuthology.orchestra.run.smithi155.stderr:real 0m9.365s 2024-03-21T13:30:39.438 INFO:teuthology.orchestra.run.smithi155.stderr:user 0m37.601s 2024-03-21T13:30:39.438 INFO:teuthology.orchestra.run.smithi155.stderr:sys 0m1.394s 2024-03-21T13:30:39.490 INFO:teuthology.orchestra.run.smithi043.stderr: 93.9% -- replaced with /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-osd.2.log.gz 2024-03-21T13:30:39.554 INFO:teuthology.orchestra.run.smithi043.stderr: 94.0% -- replaced with /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-osd.3.log.gz 2024-03-21T13:30:39.931 INFO:teuthology.orchestra.run.smithi043.stderr: 94.0% -- replaced with /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-osd.0.log.gz 2024-03-21T13:30:40.939 INFO:teuthology.orchestra.run.smithi043.stderr: 94.0% -- replaced with /var/log/ceph/c06c8d1a-e780-11ee-95cd-87774f69a715/ceph-osd.1.log.gz 2024-03-21T13:30:40.941 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-03-21T13:30:40.941 INFO:teuthology.orchestra.run.smithi043.stderr:real 0m10.871s 2024-03-21T13:30:40.941 INFO:teuthology.orchestra.run.smithi043.stderr:user 0m44.839s 2024-03-21T13:30:40.941 INFO:teuthology.orchestra.run.smithi043.stderr:sys 0m1.391s 2024-03-21T13:30:40.942 INFO:tasks.cephadm:Archiving logs... 2024-03-21T13:30:40.942 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614946/remote/smithi043/log 2024-03-21T13:30:40.943 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-21T13:30:46.439 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614946/remote/smithi155/log 2024-03-21T13:30:46.440 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-21T13:30:51.663 INFO:tasks.cephadm:Removing cluster... 2024-03-21T13:30:51.663 DEBUG:teuthology.orchestra.run.smithi043:> sudo cephadm rm-cluster --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 --force 2024-03-21T13:30:51.861 INFO:teuthology.orchestra.run.smithi043.stdout:Deleting cluster with fsid: c06c8d1a-e780-11ee-95cd-87774f69a715 2024-03-21T13:30:52.158 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm rm-cluster --fsid c06c8d1a-e780-11ee-95cd-87774f69a715 --force 2024-03-21T13:30:52.352 INFO:teuthology.orchestra.run.smithi155.stdout:Deleting cluster with fsid: c06c8d1a-e780-11ee-95cd-87774f69a715 2024-03-21T13:30:52.667 INFO:tasks.cephadm:Teardown complete 2024-03-21T13:30:52.668 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-21T13:30:52.678 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-21T13:30:52.678 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-21T13:30:52.680 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-21T13:30:52.757 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-21T13:30:52.768 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-21T13:30:52.768 DEBUG:teuthology.orchestra.run.smithi043:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T13:30:52.770 DEBUG:teuthology.orchestra.run.smithi155:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T13:30:52.782 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-03-21T13:30:52.784 INFO:teuthology.orchestra.run.smithi155.stderr:bash: line 1: ntpq: command not found 2024-03-21T13:30:52.793 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T13:30:52.793 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-03-21T13:30:52.793 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 38 +2196us[+2196us] +/- 109ms 2024-03-21T13:30:52.793 INFO:teuthology.orchestra.run.smithi043.stdout:^* hv02.front.sepia.ceph.com 3 7 377 39 -1293us[-1294us] +/- 66ms 2024-03-21T13:30:52.793 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 40 +226us[ +225us] +/- 63ms 2024-03-21T13:30:52.793 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T13:30:52.794 INFO:teuthology.orchestra.run.smithi155.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T13:30:52.794 INFO:teuthology.orchestra.run.smithi155.stdout:=============================================================================== 2024-03-21T13:30:52.794 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 39 +2110us[+2106us] +/- 109ms 2024-03-21T13:30:52.794 INFO:teuthology.orchestra.run.smithi155.stdout:^* hv02.front.sepia.ceph.com 3 6 377 39 -1392us[-1396us] +/- 66ms 2024-03-21T13:30:52.794 INFO:teuthology.orchestra.run.smithi155.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 41 +165us[ +161us] +/- 63ms 2024-03-21T13:30:52.794 INFO:teuthology.orchestra.run.smithi155.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T13:30:52.795 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-21T13:30:52.804 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-21T13:30:52.804 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-21T13:30:52.812 DEBUG:teuthology.orchestra.run.smithi043:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-21T13:30:52.917 DEBUG:teuthology.orchestra.run.smithi155:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-21T13:30:52.995 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-03-21T13:30:53.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T13:30:53.025 DEBUG:teuthology.orchestra.run.smithi155:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-03-21T13:30:53.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T13:30:53.056 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-21T13:30:53.065 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-21T13:30:53.074 INFO:teuthology.task.internal:Duration was 3219.289385 seconds 2024-03-21T13:30:53.074 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-21T13:30:53.087 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-21T13:30:53.087 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-21T13:30:53.090 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-21T13:30:53.130 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T13:30:53.139 INFO:teuthology.orchestra.run.smithi155.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T13:30:53.522 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-21T13:30:53.522 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi043.front.sepia.ceph.com 2024-03-21T13:30:53.522 DEBUG:teuthology.orchestra.run.smithi043:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-21T13:30:53.584 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi155.front.sepia.ceph.com 2024-03-21T13:30:53.585 DEBUG:teuthology.orchestra.run.smithi155:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-21T13:30:53.612 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-21T13:30:53.612 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-21T13:30:53.627 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-21T13:30:53.682 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-21T13:30:53.682 DEBUG:teuthology.orchestra.run.smithi043:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-21T13:30:53.694 DEBUG:teuthology.orchestra.run.smithi155:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-21T13:30:54.137 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-21T13:30:54.148 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-21T13:30:54.148 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-21T13:30:54.175 DEBUG:teuthology.orchestra.run.smithi155:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-21T13:30:54.203 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-21T13:30:54.213 DEBUG:teuthology.orchestra.run.smithi043:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-21T13:30:54.219 DEBUG:teuthology.orchestra.run.smithi155:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-21T13:30:54.241 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = core 2024-03-21T13:30:54.270 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = core 2024-03-21T13:30:54.299 DEBUG:teuthology.orchestra.run.smithi043:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-21T13:30:54.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T13:30:54.312 DEBUG:teuthology.orchestra.run.smithi155:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-21T13:30:54.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T13:30:54.353 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-21T13:30:54.362 INFO:teuthology.task.internal:Transferring archived files... 2024-03-21T13:30:54.363 DEBUG:teuthology.misc:Transferring archived files from smithi043:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614946/remote/smithi043 2024-03-21T13:30:54.363 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-21T13:30:54.430 DEBUG:teuthology.misc:Transferring archived files from smithi155:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614946/remote/smithi155 2024-03-21T13:30:54.430 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-21T13:30:54.470 INFO:teuthology.task.internal:Removing archive directory... 2024-03-21T13:30:54.470 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-21T13:30:54.472 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-21T13:30:54.518 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-21T13:30:54.531 INFO:teuthology.task.internal:Not uploading archives. 2024-03-21T13:30:54.531 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-21T13:30:54.543 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-21T13:30:54.544 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-21T13:30:54.546 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-21T13:30:54.557 INFO:teuthology.orchestra.run.smithi043.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 21 13:30 /home/ubuntu/cephtest 2024-03-21T13:30:54.573 INFO:teuthology.orchestra.run.smithi155.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 21 13:30 /home/ubuntu/cephtest 2024-03-21T13:30:54.574 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-21T13:30:54.583 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-21T13:30:54.602 INFO:teuthology.nuke:Checking targets against current locks 2024-03-21T13:30:54.630 DEBUG:teuthology.nuke:shortname: smithi043 2024-03-21T13:30:54.631 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T13:30:54.656 DEBUG:teuthology.nuke:shortname: smithi155 2024-03-21T13:30:54.656 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T13:30:54.660 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614946', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 12:27:01.339137', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMKQr6xJYZYRSZOKdOcsDYj9CfJ0Mdg2xHnOMm5ZlIb3Rp0OPPuqjIHStJ1jsB2FMIOxo3b96kXVOMWkJLEbG/w='} 2024-03-21T13:30:54.685 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-03-21_12:04:17-orch:cephadm-wip-guits-testing-2024-03-21-0818-distro-default-smithi/7614946', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 12:27:01.340862', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLo69eurKAR/4SxHFjnaH8P/uQ75B/82VOaeUJy+kG/DcHJuvyscQ/1DwEBzN7uTxZkbFCRdcSnhyEA9+7bdnfE='} 2024-03-21T13:30:54.707 INFO:teuthology.orchestra.console.smithi043:Power off 2024-03-21T13:30:54.708 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T13:30:54.733 DEBUG:teuthology.orchestra.console.smithi043:power off output: Chassis Power Control: Down/Off 2024-03-21T13:30:54.733 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T13:30:54.748 INFO:teuthology.orchestra.console.smithi155:Power off 2024-03-21T13:30:54.748 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T13:30:54.757 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is on 2024-03-21T13:30:54.772 DEBUG:teuthology.orchestra.console.smithi155:power off output: Chassis Power Control: Down/Off 2024-03-21T13:30:54.772 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T13:30:54.795 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2024-03-21T13:30:58.759 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T13:30:58.797 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T13:30:58.868 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is on 2024-03-21T13:30:58.905 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2024-03-21T13:31:02.873 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T13:31:02.906 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T13:31:02.983 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is off 2024-03-21T13:31:03.015 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is off 2024-03-21T13:31:03.085 INFO:teuthology.orchestra.console.smithi043:Power off completed 2024-03-21T13:31:03.116 INFO:teuthology.orchestra.console.smithi155:Power off completed 2024-03-21T13:31:03.220 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} duration: 3219.289384841919 failure_reason: '"2024-03-21T12:54:53.876180+0000 mon.a (mon.0) 1052 : cluster [WRN] Health check failed: 2 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log' flavor: default owner: scheduled_gabrioux@teuthology success: false 2024-03-21T13:31:03.220 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T13:31:03.304 INFO:teuthology.run:FAIL